|
|
基于多特征融合及背景建模的微表情Apex帧检测
|
Abstract:
微表情是人们在情感在波动时体现在面部的细微变化。根据心理学的研究，微表情在心理治疗等领域有着广泛应用，而Apex帧能够表达微表情的最丰富信息，为了更准确地提取Apex帧，本文研究了图像序列中面部微表情中的Apex帧检测问题。本文提出了一个新的检测Apex帧方法，在频率域中对面部区域采用分块方法进行背景建模，检测出脸部运动区域，随后通过统计运动区域的面积达到检测Apex帧的目的。将提出的方法应用于CASME、CASME II等数据集中，实验结果表明本文提出的方法能够有效地探测定位到Apex帧。
Micro-expressions are subtle facial changes that reflect fluctuations in emotions. According to psy-chological research, micro-expressions find widespread application in areas such as psychotherapy. Among these expressions, the Apex frame encapsulates the richest information. In order to accu-rately extract Apex frames, this study investigates the detection problem within facial mi-cro-expressions in image sequences. This paper proposes a novel method for detecting Apex frames. In the frequency domain, a block-based approach is employed for background modeling in facial regions, identifying regions of facial movement. Subsequently, the detection of Apex frames is achieved by statistically analyzing the area covered by the moving regions. The proposed method is applied to datasets such as CASME and CASME II, with experimental results demonstrating its effec-tiveness in detecting and locating Apex frames.
| [1] | Mehrabia, A. (1968) Commynication without Words. Psychology Today, 2, 53-56. |
| [2] | Li, X., Hong, X., Moilanen, A., et al. (2017) Towards Reading Hidden Emotions: A Comparative Study of Spontaneous Micro-Expression Spotting and Recognition Methods. IEEE Transactions on Affective Computing, 9, 563-577.
https://doi.org/10.1109/TAFFC.2017.2667642 |
| [3] | Frank, M.G. and Ekman, P. (1997) The Ability to Detect Deceit Gen-eralizes across Different Types of High-Stake Lies. Journal of Personality and Social Psychology, 72, 1429-1439. https://doi.org/10.1037//0022-3514.72.6.1429 |
| [4] | Xu, F. and Zhang, J.P. (2017) Facial Microexpression Recognition: A Survey. Acta Automatica Sinica, 43, 333-348. |
| [5] | Salter, F., Grammer, K. and Rikowski, A. (2005) Sex Difference in Negoti-ating with Powerful Males. Human Nature, 16, 306-321. https://doi.org/10.1007/s12110-005-1013-4 |
| [6] | Pool, L.D. and Qualter, P. (2012) Improving Emotional Intelligence and Emotional Self-Efficacy through a Teaching Intervention for University Students. Learning and Individual Differences, 22, 306-312.
https://doi.org/10.1016/j.lindif.2012.01.010 |
| [7] | Whitehill, J., Serpell, Z., Lin, Y.C., Foster, A. and Movellan, J.R. (2014) The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial Expressions. IEEE Transactions on Affec-tive Computing, 5, 86-98.
https://doi.org/10.1109/TAFFC.2014.2316163 |
| [8] | Haggard, E.A. and Isaacs, K.S. (1966) Micromomentary Facial Ex-pressions as Indicators of Ego Mechanisms in Psychotherapy. In: Haggard, E.A. and Isaacs, K.S., Eds., Methods of Research in Psychotherapy, Springer, Boston, 154-165. https://doi.org/10.1007/978-1-4684-6045-2_14 |
| [9] | Ekman, P. and Friesen, W.V. (1969) Nonverbal Leakage and Clues to Deception. Psychiatry Interpersonal & Biological Processes, 32, 88-106. https://doi.org/10.1080/00332747.1969.11023575 |
| [10] | Ekman, P. and Friesen, W.V. (1974) Nonverbal Behavior and Psychopathology. In: Friedman, R.J. and Katz, M.M., Eds., The Psychology of Depression: Contemporary Theory and Re-search, Halsted Press, Sydney, 3-31. |
| [11] | Matsumoto, D., Leroux, J., Wilson-Cohn, C., et al. (2000) A New Test to Measure Emotion Recognition Ability: Matsumoto and Ekman’s Japanese and Caucasian Brief Affect Recognition Test (JACBART). Journal of Nonverbal Behavior, 24, 179-209. https://doi.org/10.1023/A:1006668120583 |
| [12] | Ekman, P. and Friesen, W.V. (1978) Facial Action Coding System Investigator’s Guide. Consulting Psychologists Press, Palo Alto. https://doi.org/10.1037/t27734-000 |
| [13] | Ekman, P. and Friesen, W.V. (1978) Manual for the Facial Action Coding Sys-tem. Consulting Psychologists Press Palo Alto. https://doi.org/10.1037/t27734-000 |
| [14] | Pfister, T., Li, X.B., Zhao, G.Y., et al. (2011) Recognising Spontaneous Facial Micro-Expressions. 2011 International Conference on Computer Vision, Barce-lona, 6-13 November 2011, 1449-1456.
https://doi.org/10.1109/ICCV.2011.6126401 |
| [15] | Peng, M., Wang, C.Y., Chen, T., et al. (2017) Dual Temporal Scale Convolutional Neural Network for Micro-Expression Recognition. Frontiers in Psychology, 8, Article 273835. https://doi.org/10.3389/fpsyg.2017.01745 |
| [16] | Peng, M., Wu, Z., Zhang, Z.H., et al. (2018) From Macro to Micro Ex-pression Recognition: Deep Learning on Small Datasets Using Transfer Learning. 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi’an, 15-19 May 2018, 657-661. https://doi.org/10.1109/FG.2018.00103 |
| [17] | Huang, X., Wang, S.J., Zhao, G., et al. (2015) Facial Micro-Expression Recognition Using Spatiotemporal Local Binary Pattern with Integral Projection. Proceedings of the IEEE International Con-ference on Computer Vision Workshops, Santiago, Chile, 07-13 December 2015, 1-9. |
| [18] | Wang, Y.D., See, J., Phan, R.C.W., et al. (2014) LBP with Six Intersection Points: Reducing Redundant Information in LBP-Top for Micro Expression Recognition. In: Cremers, D., Reid, I., Saito, H. and Yang, M.H., Eds., Computer Vision—ACCV 2014, Springer, Cham, 525-537. https://doi.org/10.1007/978-3-319-16865-4_34 |
| [19] | Li, Y.T., Huang, X.H. and Zhao, G.Y. (2018) Can Micro-Expression Be Recognized Based on Single Apex Frame? 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, 7-10 October 2018, 3094-3098.
https://doi.org/10.1109/ICIP.2018.8451376 |
| [20] | Gan, Y.S., Liong, S.T., Yau, W.C., et al. (2019) OFF-ApexNet on Mi-cro Expressions Recognition System. Signal Processing: Image Communication, 74, 129-139. https://doi.org/10.1016/j.image.2019.02.005 |
| [21] | Patel, D., Zhao, G. and Pietik？inen, M. (2015) Spatiotemporal Integra-tion of Optical Flow Vectors for Micro-Expression Detection. In: Battiato, S., Blanc-Talon, J., Gallo, G., Philips, W., Popescu, D. and Scheunders, P., Eds., ACIVS 2015: Advanced Concepts for Intelligent Vision Systems, Springer, Cham, 369-380.
https://doi.org/10.1007/978-3-319-25903-1_32 |
| [22] | Liong, S.T., See, J., Wong, K., Le Ngo, A.C., Oh, Y.H. and Phan, R. (2015) Automatic Apex Frame Spotting in Micro- Expression Database. 2015 3rd IAPR Asian Conference on Pattern Recogni-tion (ACPR), Kuala Lumpur, 3-6 November 2015, 665-669. https://doi.org/10.1109/ACPR.2015.7486586 |
| [23] | Liong, S.T., See, J., Wong, K.S. and Pan, R.C.W. (2018) Less Is More: Micro-Expression Recognition from Video Using Apex Frame. Signal Processing: Image Communication, 62, 82-92. https://doi.org/10.1016/j.image.2017.11.006 |
| [24] | Gan, Y.S., Liong, S.T., Yau, W.C., et al. (2019) Off-Apexnet on Micro-Expression Recognition System. Signal Processing: Image Communica-tion, 74, 129-139. https://doi.org/10.1016/j.image.2019.02.005 |
| [25] | Xu, W., Zheng, H. and Yang, Z.X. (2021) Apex Frame Micro-Expression Recognition Based Dual Attention Model and Transfer Learning. CAAI Transactions on Intelligent Systems, 16, 1015-1020. |
| [26] | 温杰彬, 杨文忠, 马国详, 张志豪, 李海磊. 基于Apex帧光流和卷积自编码器的微表情识别[J]. 计算机工程与应用, 2021, 57(4): 127-133. |
| [27] | 韦锦涛, 倪建成, 侯杰, 吴俊清, 魏媛媛. 基于卷积神经网络的微表情识别模型[J]. 曲阜师范大学学报, 2021, 47(1): 64-71. |
| [28] | Zhao, Y. and Xu, J. (2019) A Convolutional Neural Net Work for Compound Micro-Expression Recognition. Sensors, 19, Article 5553. https://doi.org/10.3390/s19245553 |
| [29] | Li, Y., Huang, X. and Zhao, G. (2021) Joint Local and Global Information Learning with Single Apex Frame Detection for Micro-Expression Recognition. IEEE Transactions on Image Processing, 30, 249-263.
https://doi.org/10.1109/TIP.2020.3035042 |
| [30] | Barnich, O. and Droogenbroeck, M.V. (2011) ViBe: A Universal Back-ground Subtraction Algorithm for Video Sequences. IEEE Transactions on Image Processing, 20, 1709-1724. https://doi.org/10.1109/TIP.2010.2101613 |
| [31] | Yan, W.J., Li, X., Wang, S.J., et al. (2014) CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation. PLOS ONE, 9, e86041. https://doi.org/10.1371/journal.pone.0086041 |
| [32] | Liong, S.T., See, J., Wong, K.S., et al. (2015) Automatic Apex Frame Spotting in Micro-Expression Database. 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), Kuala Lumpur, 3-6 November 2015, 665-669.
https://doi.org/10.1109/ACPR.2015.7486586 |
| [33] | Yan, W.J., Wang, S.J., Chen, Y.H., et al. (2015) Quantifying Mi-cro-Expressions with Constraint Local Model and Local Binary Pattern. In: Agapito, L., Bronstein, M. and Rother, C., Eds., Computer Vision—ECCV 2014 Workshops, Springer, Cham, 296-305. https://doi.org/10.1007/978-3-319-16178-5_20 |
| [34] | Liong, S.T., Phan, C.W., See, J., et al. (2015) Optical Strain Based Recognition of Subtle Emotions. 2014 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), Kuching, 1-4 December 2014, 180-184. https://doi.org/10.1109/ISPACS.2014.7024448 |
| [35] | 马浩原. 微表情特征提取算法研究[D]: [硕士学位论文]. 北京: 北京交通大学, 2018. |
