Facial expressions are the straight link for showing human emotions. Psychologists have established the universality of six prototypic basic facial expressions of emotions which they believe are consistent among cultures and races. However, some recent cross-cultural studies have questioned and to some degree refuted this cultural universality. Therefore, in order to contribute to the theory of cultural specificity of basic expressions, from a composite viewpoint of psychology and HCI (Human Computer Interaction), this paper presents a methodical analysis of Western-Caucasian and East-Asian prototypic expressions focused on four facial regions: forehead, eyes-eyebrows, mouth and nose. Our analysis is based on facial expression recognition and visual analysis of facial expression images of two datasets composed by four standard databases CK+, JAFFE, TFEID and JACFEE. A hybrid feature extraction method based on Fourier coefficients is proposed for the recognition analysis. In addition, we present a cross-cultural human study applied to 40 subjects as a baseline, as well as one comparison of facial expression recognition performance between the previous cross-cultural tests from the literature. With this work, it is possible to clarify the prior considerations for working with multicultural facial expression recognition and contribute to identifying the specific facial expression differences between Western-Caucasian and East-Asian basic expressions of emotions.
References
[1]
Darwin, C. (1872) The Expression of the Emotions in Man and Animals. Oxford University Press, Oxford. https://doi.org/10.1037/10001-000
[2]
Ekman, P. (1972) Universal and Cultural Differences in Facial Expression of Emotion. In: Proceedings of Symposium on Motivation, Nebraska University Press, Lincoln, 9-15.
[3]
Ekman, P., Friesen, W.V. and Hager, J.C. (1978) Facial Action Coding System (FACS). A Technique for the Measurement of Facial Action. Consulting, Palo Alto, 22.
[4]
Jack, R.E. (2013) Culture and Facial Expressions of Emotion. Visual Cognition, 21, 1248-1286. https://doi.org/10.1080/13506285.2013.835367
[5]
Dailey, M.N., et al. (2010) Evidence and a Computational Explanation of Cultural Differences in Facial Expression Recognition. Emotion, 10, 874-893.
https://doi.org/10.1037/a0020019
[6]
Jack, R.E., Garrod, O.G., Yu, H., Caldara, R. and Schyns, P.G. (2012) Facial Expressions of Emotion Are Not Culturally Universal. Proceedings of the National Academy of Sciences, 109, 7241-7244. https://doi.org/10.1073/pnas.1200155109
[7]
Tian, Y., Kanade, T. and Cohn, J.F. (2011) Facial Expression Recognition. In: Handbook of Face Recognition, Springer, London, 487-519.
https://doi.org/10.1007/978-0-85729-932-1_19
[8]
Sariyanidi, E., Gunes, H. and Cavallaro, A. (2015) Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37, 1113-1133.
https://doi.org/10.1109/TPAMI.2014.2366127
[9]
Deshmukh, S., Patwardhan, M. and Mahajan, A. (2016) Survey on Real-Time Facial Expression Recognition Techniques. IET Biometrics, 6, 155-163.
[10]
Li, Z., Imai, J.I. and Kaneko, M. (2010) Facial Expression Recognition Using Facial-Component-Based Bag of Words and PHOG Descriptors. Journal of ITE, 64, 230-236. https://doi.org/10.3169/itej.64.230
[11]
Gu, W., Xiang, C., Venkatesh, Y.V., Huang, D. and Lin, H. (2012) Facial Expression Recognition Using Radial Encoding of Local Gabor Features and Classifier Synthesis. Pattern Recognition, 45, 80-91.
[12]
Shan, C., Gong, S. and McOwan, P.W. (2009) Facial Expression Recognition Based on Local Binary Patterns: A Comprehensive Study. Image and Vision Computing, 27, 803-816.
[13]
Ali, G., Iqbal, M.A. and Choi, T.S. (2016) Boosted NNE Collections for Multicultural Facial Expression Recognition. Pattern Recognition, 55, 14-27.
[14]
Lopes, A.T., de Aguiar, E., De Souza, A.F. and Oliveira-Santos, T. (2017) Facial Expression Recognition with Convolutional Neural Networks: Coping with Few Data and the Training Sample Order. Pattern Recognition, 61, 610-628.
[15]
Hwang, W., Wang, H., Kim, H., Kee, S.C. and Kim, J. (2011) Face Recognition System Using Multiple Face Model of Hybrid Fourier Feature under Uncontrolled Illumination Variation. IEEE Transactions on Image Processing, 20, 1152-1165.
https://doi.org/10.1109/TIP.2010.2083674
[16]
Benitez-Garcia, G., Olivares-Mercado, J., Sanchez-Perez, G., Nakano-Miyatake, M. and Perez-Meana, H. (2013) A Sub-Block-Based Eigenphases Algorithm with Optimum Sub-Block Size. Knowledge-Based Systems, 37, 415-426.
[17]
Benitez-Garcia, G., Sanchez-Perez, G., Perez-Meana, H., Takahashi, K. and Kaneko, M. (2014) Facial Expression Recognition Based on Facial Region Segmentation and Modal Value Approach. IEICE Transactions on Information and Systems, 97, 928-935. https://doi.org/10.1587/transinf.E97.D.928
[18]
Turk, M.A. and Pentland, A.P. (1991) Face Recognition Using Eigenfaces. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Hawaii, 3-6 June 1991, 586-591. https://doi.org/10.1109/cvpr.1991.139758
[19]
Benitez-Garcia, G., Nakamura, T. and Kaneko, M. (2017) Analysis of Differences between Western and East-Asian Faces Based on Facial Region Segmentation and PCA for Facial Expression Recognition. In: AIP Conference Proceedings 1807, AIP Publishing, Tokyo, Article ID: 020025. https://doi.org/10.1063/1.4974807
[20]
Da Silva, F.A.M. and Pedrini, H. (2015) Effects of Cultural Characteristics on Building an Emotion Classifier through Facial Expression Analysis. Journal of Electronic Imaging, 24, 0230151-0230159. https://doi.org/10.1117/1.jei.24.2.023015
[21]
Benitez-Garcia, G., Nakamura, T. and Kaneko, M. (2017) Analysis of In- and Out-Group Differences between Western and East-Asian Facial Expression Recognition. Proceedings of the 14th IAPR International Conference on Machine Vision Applications (MVA 2017), Nagoya, 8-12 May 2017. (In Press)
[22]
El-Bakry, H.M. (2001) Automatic Human Face Recognition Using Modular Neural Networks. Machine Graphics and Vision, 10, 47-73.
[23]
Yi, J., Mao, X., Chen, L., Xue, Y. and Compare, A. (2014) Facial Expression Recognition Considering Individual Differences in Facial Structure and Texture. IET Computer Vision, 8, 429-440. https://doi.org/10.1049/iet-cvi.2013.0171
[24]
Mohammadi, M.R., Fatemizadeh, E. and Mahoor, M.H. (2014) PCA-Based Dictionary Building for Accurate Facial Expression Recognition via Sparse Representation. Journal of Visual Communication and Image Representation, 25, 1082-1092.
[25]
Calder, A.J., Burton, A.M., Miller, P., Young, A.W. and Akamatsu, S. (2001) A Principal Component Analysis of Facial Expressions. Vision Research, 41, 1179-1208.
[26]
Kaneko, M. (2008) Computerized Facial Caricatures. Journal of Institute of Image Information and Television Engineers, 62, 1938-1943.
https://doi.org/10.3169/itej.62.1938
[27]
Kanade, T., Cohn, J.F. and Tian, Y. (2000) Comprehensive Database for Facial Expression Analysis. Proceedings of 4th IEEE International Conference on Automatic Face and Gesture Recognition, Washington DC, 28-30 March 2000, 46-53.
https://doi.org/10.1109/AFGR.2000.840611
[28]
Lyons, M., Akamatsu, S., Kamachi, M. and Gyoba, J. (1998) Coding Facial Expressions with Gabor Wavelets. Proceedings of 3rd IEEE International Conference on Automatic Face and Gesture Recognition, Nara, 14-16 April 1998, 200-205.
https://doi.org/10.1109/AFGR.1998.670949
[29]
Biehl, M., et al. (1997) Matsumoto and Ekman’s Japanese and Caucasian Facial Expressions of Emotion (JACFEE): Reliability Data and Cross-National Differences. Journal of Nonverbal Behavior, 21, 3-21. https://doi.org/10.1023/A:1024902500935
[30]
Chen, L.F. and Yen, Y.S. (2007) Taiwanese Facial Expression Image Database. Brain Mapping Laboratory, Institute of Brain Science, National Yang-Ming University, Taipei.
[31]
Chang, C.C. and Lin, C.J. (2011) LIBSVM: A Library for Support Vector Machines. ACM Transactions on Intelligent Systems and Technology, 2, 27.
https://doi.org/10.1145/1961189.1961199
