|
|
基于多尺度先验融合的小样本语义分割方法
|
Abstract:
小样本语义分割侧重于模型泛化能力的提升，用有限的样本为未见类提供像素级分割。现有方法在分割性能上已经取得了巨大的进步，但仍受到部分因素的限制，例如当同类物体中因较大的角度、尺寸等差异，存在类内差异时，将导致模型难以捕捉类别语义。因此，本文提出基于多尺度先验信息融合的小样本语义分割模块，通过提取高级特征中的多尺度先验掩码，为模型补充类别相关信息，降低类内差异的影响。为了融合多尺度特征及其先验信息，本文提出权重自适应特征融合模块，为各尺度添加信息交互，并加权组合特征。最终通过消融实验以及与基类方法和其他经典算法的对比实验，证明了本方法的有效性。
Few-shot semantic segmentation focuses on improving the model’s generalization ability to provide pixel-level segmentation for unseen classes with limited samples. While existing methods have made significant progress in segmentation performance, they are still constrained by certain fac-tors. For instance, when there are intra-class gaps due to significant variations in angles, sizes, etc., within the same class, the model struggles to capture category semantics. Therefore, this paper proposes a few-shot semantic segmentation module based on the fusion of multi-scale prior infor-mation. By extracting multi-scale prior masks from high-level features, this module supplements category-related information for the model, reducing the impact of intra-class differences. To inte-grate multi-scale features and their prior information, the paper introduces a Weighted Adaptive Feature Fusion module, adding information interaction across various scales and combining features with weighted contributions. Ultimately, through ablation experiments and comparative ex-periments with baseline methods and other classical algorithms, the effectiveness of this approach has been demonstrated.
| [1] | Wu, Z., Shi, X., Lin, G. and Cai, J. (2021) Learning Meta-Class Memory for Few-Shot Semantic Segmentation. Pro-ceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, 10-17 October 2021, 517-526.
https://doi.org/10.1109/ICCV48922.2021.00056 |
| [2] | Zhang, X., Wei, Y., Yang, Y. and Huang, T.S. (2020) SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation. IEEE Transactions on Cybernetics, 50, 3855-3865. https://doi.org/10.1109/TCYB.2020.2992433 |
| [3] | Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., et al. (2015) ImageNet Large Scale Visual Recognition Challenge. International Journal of Com-puter Vision, 115, 211-252. https://doi.org/10.1007/s11263-015-0816-y |
| [4] | Zhang, C., Lin, G., Liu, F., Yao, R. and Shen, C. (2019) CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15-20 June 2019, 5217-5226. https://doi.org/10.1109/CVPR.2019.00536 |
| [5] | Wang, K., Liew, J.H., Zou, Y., Zhou, D. and Feng, J. (2019) PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment. Proceedings of the IEEE/CVF Inter-national Conference on Computer Vision, Seoul, 27 October 2019 - 2 November 2019, 9197-9206. https://doi.org/10.1109/ICCV.2019.00929 |
| [6] | Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K. and Yuille, A.L. (2017) DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 834-848. https://doi.org/10.1109/TPAMI.2017.2699184 |
| [7] | Tian, Z., Zhao, H., Shu, M., Yang, Z., Li, R. and Jia, J. (2020) Prior Guided Feature Enrichment Network for Few-Shot Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44, 1050-1065.
https://doi.org/10.1109/TPAMI.2020.3013717 |
| [8] | He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 770-778.
https://doi.org/10.1109/CVPR.2016.90 |
| [9] | Liu, W., Zhang, C., Lin, G. and Liu, F. (2020) CRNet: Cross-Reference Networks for Few-Shot Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 13-19 June 2020, 4165-4173.
https://doi.org/10.1109/CVPR42600.2020.00422 |
| [10] | Liu, Y., Zhang, X., Zhang, S. and He, X. (2020) Part-Aware Prototype Network for Few-Shot Semantic Segmentation. Computer Vision—ECCV 2020: 16th European Conference, Glasgow, 23-28 August 2020, 142-158.
https://doi.org/10.1007/978-3-030-58545-7_9 |
| [11] | Lu, Z., He, S., Zhu, X., Zhang, L., Song, Y.Z. and Xiang, T. (2021) Simpler Is Better: Few-Shot Semantic Segmentation with Classifier Weight Transformer. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, 10-17 October 2021, 8741-8750. https://doi.org/10.1109/ICCV48922.2021.00862 |
| [12] | Yang, X., Wang, B., Chen, K., Zhou, X., Yi, S., Ouyang, W. and Zhou, L. (2020) Brinet: Towards Bridging the Intra-Class and Inter-Class Gaps One-Shot Segmentation. arXiv: 2008.06226. |
| [13] | Li, G., Jampani, V., Sevilla-Lara, L., Sun, D., Kim, J. and Kim, J. (2021) Adaptive Prototype Learning and Allocation for Few-Shot Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, 20-25 June 2021, 8334-8343. https://doi.org/10.1109/CVPR46437.2021.00823 |
| [14] | Yang, B., Liu, C., Li, B., Jiao, J. and Ye, Q. (2020) Prototype Mixture Models for Few-Shot Semantic Segmentation. Computer Vision—ECCV 2020: 16th European Conference, Glasgow, 23-28 August 2020, 763-778.
https://doi.org/10.1007/978-3-030-58598-3_45 |
| [15] | Liu, B., Jiao, J. and Ye, Q. (2021) Harmonic Feature Activa-tion for Few-Shot Semantic Segmentation. IEEE Transactions on Image Processing, 30, 3142-3153. https://doi.org/10.1109/TIP.2021.3058512 |
| [16] | Liu, Y., Liu, N., Cao, Q., Yao, X., Han, J. and Shao, L. (2022) Learning Non-Target Knowledge for Few-Shot Semantic Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, 18-24 June 2022, 11573-11582. https://doi.org/10.1109/CVPR52688.2022.01128 |
