|
|
基于改进的MUNIT人脸图像性别转换模型
|
Abstract:
基于生成对抗网络的图像风格迁移算法已成为人脸图像性别转换的主流模型，但现有方法仍存在转化后的人脸图像模糊，背景图像扭曲，面部身份保留效果不好等缺点。针对上述问题，基于多模态无监督图像翻译网络(MUNIT)，本文提出了基于改进的人脸图像性别转换模型。首先对MUNIT模型生成器部分进行优化，在编码器部分加入动态实例归一化操作(DIN)，使编码器对人脸内容特征和风格特征的剥离更加精确；并在内容编码部分的残差块网络后加入混合注意力模块(CBAM)，使模型提取更丰富的人脸关键特征；此外，对CeleBA数据集的人脸图像根据属性进行筛选以及裁剪，减少了图像背景对于生成图像的影响，使模型更加专注于人脸特征的学习。根据实验对照情况，本文方法能够生成更加精细的人脸性别转换图像。
The image style transfer methods based on generative adversarial network have become the main-stream model of face image gender transformation. However, for existing methods, the transformed face image is blurred, the background image is distorted, and the facial identity preservation effect is not good. Aiming at the above problems, this paper proposes an improved face image gender transformation model based on multi-modal unsupervised Image Translation Network (MUNIT). Firstly, the generator part of MUNIT model is optimized, and the dynamic instance normalization operation (DIN) is added to the encoder part to make the encoder more accurate in the stripping of face content features and style features. The mixed attention module (CBAM) is added after the residual block network in the content encoding part, so that the model can extract more abundant face key features. In addition, the face image of CeleBA dataset is screened and trimmed according to its attributes, which reduces the influence of image background on image generation and makes the model more focused on the learning of face features. According to the experimental situation, the proposed method can generate more refined facial gender conversion images.
| [1] | Schroff, F., Kalenichenko, D. and Philbin, J. (2015) FaceNet: A Unified Embedding for Face Recognition and Clustering. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, 7-12 June 2015, 815-823.
https://doi.org/10.1109/CVPR.2015.7298682 |
| [2] | An, X., Deng, J., Guo, J., Feng, Z., Zhu, X., Yang, J., et al. (2022) Killing Two Birds with One Stone: Efficient and Robust Training of Face Recognition CNNs by Partial FC. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, 18-24 June 2022, 4032-4041.
https://doi.org/10.1109/CVPR52688.2022.00401 |
| [3] | Zhang, K., Zhang, Z., Li, Z. and Qiao, Y. (2016) Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks. IEEE Signal Processing Letters, 23, 1499-1503. https://doi.org/10.1109/LSP.2016.2603342 |
| [4] | Sun, K., Wu, W., Liu, T., Yang, S., Wang, Q., Zhou, Q., Ye, Z. and Qian, C. (2019) FAB: A Robust Facial Landmark Detection Framework for Motion-Blurred Videos. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, 27 October-2 November 2019, 5461-5470. https://doi.org/10.1109/ICCV.2019.00556 |
| [5] | Feng, Y., Wu, F., Shao, X., Wang, Y. and Zhou, X. (2018) Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network. 15th European Conference, Mu-nich, 8-14 September 2018, 557-574.
https://doi.org/10.1007/978-3-030-01264-9_33 |
| [6] | Nguyen, T., Tran, A. and Hoai, M. (2021) Lipstick Ain’t Enough: Beyond Color Matching for In-the-Wild Makeup Transfer. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, 20-25 June 2021, 13300-13309. https://doi.org/10.1109/CVPR46437.2021.01310 |
| [7] | Sun, Z., Chen, Y. and Xiong, S. (2022) SSAT: A Symmet-ric Semantic-Aware Transformer Network for Makeup Transfer and Removal. Proceedings of the AAAI Conference on Artificial Intelligence, 36, 2325-2334.
https://doi.org/10.1609/aaai.v36i2.20131 |
| [8] | Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A.C. and Bengio, Y. (2014) Generative Adversarial Nets. Proceedings of the NIPS 2014 Workshop on High-Energy Physics and Machine Learning, Montreal, 13 December 2014, 2672-2680. |
| [9] | Karras, T., Laine, S. and Aila, T. (2019) A Style-Based Generator Architecture for Generative Adver-sarial Networks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 4396-4405. https://doi.org/10.1109/CVPR.2019.00453 |
| [10] | Rai, H. and Shukla, N. (2018) Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. |
| [11] | Kim, J., Kim, M., Kang, H. and Lee, K. (2020) U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Im-age-to-Image Translation. |
| [12] | Huang, X., Liu, M., Belongie, S.J. and Kautz, J. (2018) Multimodal Unsupervised Im-age-to-Image Translation. ECCV 2018: 15th European Conference, Munich, 8-14 September 2018, 179-196.
https://doi.org/10.1007/978-3-030-01219-9_11 |
| [13] | Jing, Y., Liu, X., Ding, Y., Wang, X., Ding, E., Song, M. and Wen, S. (2020) Dynamic Instance Normalization for Arbitrary Style Transfer. |
| [14] | Woo, S., Park, J., Lee, J. and Kweon, I. (2018) CBAM: Convolutional Block Attention Module. ECCV 2018: 15th European Conference, Munich, 8-14 Sep-tember 2018, 3-19. https://doi.org/10.1007/978-3-030-01234-2_1 |
| [15] | Liu, M., Breuel, T.M. and Kautz, J. (2017) Unsupervised Image-to-Image Translation Networks. |
| [16] | Isola, P., Zhu, J., Zhou, T. and Efros, A.A. (2017) Im-age-to-Image Translation with Conditional Adversarial Networks. 2017 IEEE Conference on Computer Vision and Pat-tern Recognition (CVPR), Honolulu, 21-26 July 2017, 5967-5976. https://doi.org/10.1109/CVPR.2017.632 |
| [17] | Mirza, M. and Osindero, S. (2014) Conditional Generative Adver-sarial Nets. |
| [18] | Ma, Z., Li, J., Wang, N. and Gao, X. (2020) Semantic-Related Image Style Transfer with Du-al-Consistency Loss. Neurocomputing, 406, 135-149. https://doi.org/10.1016/j.neucom.2020.04.027 |
| [19] | Choi, Y., Choi, M., Kim, M.S., Ha, J., Kim, S. and Choo, J. (2018) StarGAN: Unified Generative Adversarial Networks for Mul-ti-Domain Image-to-Image Translation. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-22 June 2018, 8789-8797. https://doi.org/10.1109/CVPR.2018.00916 |
| [20] | Sanakoyeu, A., Kotov-enko, D., Lang, S. andOmmer, B. (2018) A Style-Aware Content Loss for Real-Time HD Style Transfer. ECCV 2018: 15th European Conference, Munich, 8-14 September 2018, 715-731.
https://doi.org/10.1007/978-3-030-01237-3_43 |
| [21] | Wu, H., Liu, Q., Wang, Y., Mathematics, S.O. and University, T. (2019) Face Image Translation Based on Generative Adversarial Networks. Journal of Tianjin University (Science and Technology), 52, 306-314. |
| [22] | Peng, Y.F., Wang, K.X., Mei, J.Y., et al. (2020) Image Style Migration Based on Cycle Generative Adversarial Net Works. Computer Engineering & Science, 42, 699-706. |
| [23] | Bao, J., Chen, D., Wen, F., Li, H. and Hua, G. (2017) CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training. 2017 IEEE Inter-national Conference on Computer Vision (ICCV), Venice, 22-29 October 2017, 2764-2773. https://doi.org/10.1109/ICCV.2017.299 |
| [24] | 石达, 芦天亮, 杜彦辉, 张建岭, 暴雨轩. 基于改进CycleGAN的人脸性别伪造图像生成模型[J]. 计算机科学, 2022, 49(2): 31-39. |
| [25] | Liu, X., Wang, R., Peng, H., Yin, M., Chen, C. and Li, X. (2021) Sparse Feature Representation Learning for Deep Face Gender Transfer. 2021 IEEE/CVF Interna-tional Conference on Computer Vision Workshops (ICCVW), Montreal, 11-17 October 2021, 4070-4080. https://doi.org/10.1109/ICCVW54120.2021.00454 |
| [26] | Jaderberg, M., Simonyan, K., Zisserman, A. and Kavukcuoglu, K. (2015) Spatial Transformer Networks. NIPS 2015, Montreal, 5 June 2015, 2017-2025. |
| [27] | Hu, J., Shen, L., Albanie, S., Sun, G. and Wu, E. (2020) Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 2011-2023. https://doi.org/10.1109/TPAMI.2019.2913372 |
| [28] | Wang, T., Liu, M., Zhu, J., Tao, A., Kautz, J. and Catanzaro, B. (2018) High-Resolution Image Synthesis and Semantic Manipula-tion with Conditional GANs. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-22 June 2018, 8798-8807. https://doi.org/10.1109/CVPR.2018.00917 |
| [29] | Kingma, D.P. and Ba, J. (2015) Adam: A Method for Stochastic Optimization. |
| [30] | Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. and Wojna, Z. (2016) Rethinking the Inception Architecture for Computer Vision. 2016 IEEE Conference on Computer Vision and Pat-tern Recognition (CVPR), Las Vegas, 27-30 June 2016, 2818-2826. https://doi.org/10.1109/CVPR.2016.308 |
