|
|
图像超分辨率重建综述
|
Abstract:
图像超分辨率重建是计算机视觉领域中一个备受关注的研究方向，其目标是通过使用先进的超分辨率方法，将低分辨率图像提升至高分辨率，以改善图像质量和细节。图像超分辨率重建在医学影像、计算机视觉和卫星遥感等领域具有广泛的应用。本文涵盖了基于深度学习的单图像超分辨率和多图像超分辨率的发展历程和最新进展，并探讨了两类方法的优势与局限性。图像超分辨率重建仍然充满挑战和机遇，文章最后展望了图像超分辨率重建的未来研究方向。
Image super-resolution reconstruction is a popular research field in computer vision. Its goal is to improve image quality and detail by using advanced super-resolution methods to elevate low-resolution images to high resolution. Image super-resolution reconstruction is widely used in medical imaging, computer vision and satellite remote sensing. This paper covers the development and latest progress of single image super-resolution and multi-image super-resolution based on deep learning, and discusses the advantages and limitations of the two types of methods. Image super-resolution reconstruction is still full of challenges and opportunities. Finally, the future re-search direction of image super-resolution reconstruction is discussed.
| [1] | Chen, H., He, X., Qing, L., et al. (2022) Real-World Single Image Super-Resolution: A Brief Review. Information Fu-sion, 79, 124-145. https://doi.org/10.1016/j.inffus.2021.09.005 |
| [2] | Lepcha, D.C., Goyal, B., Dogra, A., et al. (2023) Image Super-Resolution: A Comprehensive Review, Recent Trends, Challenges and Applications. Information Fusion, 91, 230-260. https://doi.org/10.1016/j.inffus.2022.10.007 |
| [3] | Wang, P., Bayram, B. and Sertel, E. (2022) A Comprehensive Review on Deep Learning Based Remote Sensing Image Super-Resolution Methods. Earth-Science Reviews, 2022, Article ID: 104110.
https://doi.org/10.1016/j.earscirev.2022.104110 |
| [4] | Wang, L., Li, D., Zhu, Y., et al. (2020) Dual Su-per-Resolution Learning for Semantic Segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 14-19 June 2020, 3774-3783.
https://doi.org/10.1109/CVPR42600.2020.00383 |
| [5] | Chen, M.J., Huang, C.H. and Lee, W.L. (2005) A Fast Edge-Oriented Algorithm for Image Interpolation. Image and Vision Computing, 23, 791-798. https://doi.org/10.1016/j.imavis.2005.05.005 |
| [6] | Tom, B.C. and Katsaggelos, A.K. (1995) Reconstruction of a High-Resolution Image by Simultaneous Registration, Restoration, and Interpolation of Low-Resolution Images. Pro-ceedings IEEE International Conference on Image Processing, Vol. 2, 539-542. |
| [7] | Wang, Z., Chen, J. and Hoi, S.C.H. (2020) Deep Learning for Image Super-Resolution: A Survey. IEEE Transactions on Pattern Analysis and Ma-chine Intelligence, 43, 3365-3387. https://doi.org/10.1109/TPAMI.2020.2982166 |
| [8] | Yang, W., Zhang, X., Tian, Y., et al. (2019) Deep Learning for Single Image Super-Resolution: A Brief Review. IEEE Transactions on Multimedia, 21, 3106-3121. https://doi.org/10.1109/TMM.2019.2919431 |
| [9] | Liu, A., Liu, Y., Gu, J., et al. (2022) Blind Im-age Super-Resolution: A Survey and Beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 5461-5480. https://doi.org/10.1109/TPAMI.2022.3203009 |
| [10] | Arefin, M.R., Michalski, V., St-Charles, P.L., et al. (2020) Multi-Image Super-Resolution for Remote Sensing Using Deep Recurrent Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, 14-19 June 2020, 206-207. |
| [11] | Dong, C., Loy, C.C., He, K., et al. (2014) Learning a Deep Convolutional Network for Image Su-per-Resolution. In: European Conference on Computer Vision, Springer, Cham, 184-199. https://doi.org/10.1007/978-3-319-10593-2_13 |
| [12] | Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012) Imagenet Classification with Deep Convolutional Neural Networks. 26th Annual Conference on Neural Information Processing Systems, Lake Tahoe, 3-6 December 2012, 1097-1105. |
| [13] | Liu, J., Zou, M., Tang, J., et al. (2020) Memory Recursive Network for Single Image Super-Resolution. Proceedings of the 28th ACM International Conference on Mul-timedia, Seattle, 12-16 October 2020, 2202-2210.
https://doi.org/10.1145/3394171.3413696 |
| [14] | Liu, F., Yang, X. and De Baets, B. (2023) A Deep Recursive Mul-ti-Scale Feature Fusion Network for Image Super-Resolution. Journal of Visual Communication and Image Representa-tion, 90, Article ID: 103730.
https://doi.org/10.1016/j.jvcir.2022.103730 |
| [15] | He, K., Zhang, X., Ren, S., et al. (2016) Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 770-778.
https://doi.org/10.1109/CVPR.2016.90 |
| [16] | Kim, J., Lee, J.K. and Lee, K.M. (2016) Accurate Image Su-per-Resolution Using Very Deep Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 1646-1654. https://doi.org/10.1109/CVPR.2016.182 |
| [17] | Li, Z., Liu, Y., Chen, X., et al. (2022) Blueprint Separable Residual Network for Efficient Image Super-Resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pat-tern Recognition, New Orleans, 18-24 June 2022, 833-843. https://doi.org/10.1109/CVPRW56347.2022.00099 |
| [18] | Gendy, G., Sabor, N., Hou, J., et al. (2023) Mixer-Based Local Residual Network for Lightweight Image Super-Resolution. Proceedings of the IEEE/CVF Conference on Com-puter Vision and Pattern Recognition, Vancouver, 18-22 June 2023, 1593-1602. https://doi.org/10.1109/CVPRW59228.2023.00161 |
| [19] | Song, D., Xu, C., Jia, X., et al. (2020) Efficient Residual Dense Block Search for Image Super-Resolution. Proceedings of the AAAI Conference on Artificial Intelligence, 34, 12007-12014. https://doi.org/10.1609/aaai.v34i07.6877 |
| [20] | Tong, T., Li, G., Liu, X., et al. (2017) Image Su-per-Resolution Using Dense Skip Connections. Proceedings of the IEEE International Conference on Computer Vision, Venice, 22-29 October 2017, 4799-4807.
https://doi.org/10.1109/ICCV.2017.514 |
| [21] | Lv, X., Wang, C., Fan, X., et al. (2022) A Novel Image Su-per-Resolution Algorithm Based on Multi-Scale Dense Recursive Fusion Network. Neurocomputing, 489, 98-111. https://doi.org/10.1016/j.neucom.2022.02.042 |
| [22] | Tian, C., Zhang, Y., Zuo, W., et al. (2022) A Heterogeneous Group CNN for Image Super-Resolution. IEEE Transactions on Neural Networks and Learning Sys-tems. |
| [23] | Ruangsang, W., Aramvith, S. and Onoye, T. (2023) Multi-FusNet of Cross Channel Network for Image Su-per-Resolution. IEEE Access, 11, 56287-56299. https://doi.org/10.1109/ACCESS.2023.3282571 |
| [24] | Li, Y., Iwamoto, Y., Lin, L., et al. (2020) Parallel-Connected Residual Channel Attention Network for Remote Sensing Image Super-Resolution. Proceedings of the Asian Conference on Computer Vision, Kyoto, 30 November 2020 - 4 December 2020, 18-30. |
| [25] | Yang, Y. and Qi, Y. (2021) Image Super-Resolution via Channel Attention and Spatial Graph Con-volutional Network. Pattern Recognition, 112, Article ID: 107798. https://doi.org/10.1016/j.patcog.2020.107798 |
| [26] | Zhang, X., Zeng, H., Guo, S., et al. (2022) Efficient Long-Range Attention Network for Image Super-Resolution. In: European Conference on Computer Vision, Springer Nature, Cham, 649-667.
https://doi.org/10.1007/978-3-031-19790-1_39 |
| [27] | Yoo, J., Kim, T., Lee, S., et al. (2023) Enriched CNN-Transformer Feature Aggregation Networks for Super-Resolution. Proceedings of the IEEE/CVF Winter Confer-ence on Applications of Computer Vision, Waikoloa, 3-7 January 2023, 4956-4965. https://doi.org/10.1109/WACV56688.2023.00493 |
| [28] | Wang, Z., Zhang, Z., Zhang, X., et al. (2023) DR2: Diffu-sion-Based Robust Degradation Remover for Blind Face Restoration. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, 18-22 June 2023, 1704-1713. https://doi.org/10.1109/CVPR52729.2023.00170 |
| [29] | Zhang, K., Gool, L.V. and Timofte, R. (2020) Deep Un-folding Network for Image Super-Resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pat-tern Recognition, Seattle, 14-19 June 2020, 3217-3226.
https://doi.org/10.1109/CVPR42600.2020.00328 |
| [30] | Chen, X., Zhang, J., Xu, C., et al. (2023) Better “CMOS” Produces Clearer Images: Learning Space-Variant Blur Estimation for Blind Image Super-Resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, 18-22 June 2023, 1651-1661. https://doi.org/10.1109/CVPR52729.2023.00165 |
| [31] | Lee, R., Li, R., Venieris, S., et al. (2024) Meta-Learned Kernel for Blind Super-Resolution Kernel Estimation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, 4-8 January 2024, 1496-1505. |
| [32] | Wei, Y., Gu, S., Li, Y., et al. (2021) Unsupervised Real-World Image Super Resolution via Domain-Distance Aware Training. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19-25 June 2021, 13385-13394. https://doi.org/10.1109/CVPR46437.2021.01318 |
| [33] | Zhou, H., Zhu, X., Zhu, J., et al. (2023) Learning Correction Filter via Degradation-Adaptive Regression for Blind Single Image Super-Resolution. Proceedings of the IEEE/CVF In-ternational Conference on Computer Vision, Paris, 2-6 October 2023, 12365-12375. https://doi.org/10.1109/ICCV51070.2023.01136 |
| [34] | Weng, S.Y., Yuan, H., Xu, Y.S., et al. (2024) Best of both Worlds: Learning Arbitrary-Scale Blind Super-Resolution via Dual Degradation Representations and Cycle-Consistency. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, 4-8 January 2024, 1547-1556. |
| [35] | Zanotta, D.C., Junior, A.M., Motta, J.G., et al. (2023) An Assisted Multi-Frame Approach for Su-per-Resolution in Hyperspectral Images of Rock Samples. Computers & Geosciences, 181, Article ID: 105456.
https://doi.org/10.1016/j.cageo.2023.105456 |
| [36] | Lu, L., Li, W., Tao, X., et al. (2021) Masa-Sr: Matching Acceler-ation and Spatial Adaptation for Reference-Based Image Super-Resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19-25 June 2021, 6368-6377. https://doi.org/10.1109/CVPR46437.2021.00630 |
| [37] | Ibrahim, M.R., Benavente, R., Lumbreras, F., et al. (2022) 3DRRDB: Super Resolution of Multiple Remote Sensing Images Using 3D Residual in Residual Dense Blocks. Pro-ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, 18-24 June 2022, 323-332. https://doi.org/10.1109/CVPRW56347.2022.00047 |
| [38] | Tsai, R.Y. and Huang, T.S. (1984) Multiframe Image Restoration and Registration. In: Huang, T.S., Ed., Advances in Computer Vision and Image Processing, JAI Press Inc., Greenwich, 317-339. |
| [39] | Bhat, G., Danelljan, M., Yu, F., et al. (2021) Deep Reparametrization of Mul-ti-Frame Super-Resolution and Denoising. Proceedings of the IEEE/CVF International Conference on Computer Vision, 11-17 October 2021, 2460-2470.
https://doi.org/10.1109/ICCV48922.2021.00246 |
| [40] | Mehta, N., Dudhane, A., Murala, S., et al. (2022) Adaptive Feature Consolidation Network for Burst Super-Resolution. Proceedings of the IEEE/CVF Conference on Computer Vi-sion and Pattern Recognition, New Orleans, 18-24 June 2022, 1279-1286. https://doi.org/10.1109/CVPRW56347.2022.00134 |
| [41] | Wei, P., Sun, Y., Guo, X., et al. (2023) Towards Re-al-World Burst Image Super-Resolution: Benchmark and Method. Proceedings of the IEEE/CVF International Confer-ence on Computer Vision, Paris, 2-6 October 2023, 13233-13242.
https://doi.org/10.1109/ICCV51070.2023.01217 |
| [42] | Zhang, Z., Wang, Z., Lin, Z., et al. (2019) Image Su-per-Resolution by Neural Texture Transfer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 7982-7991.
https://doi.org/10.1109/CVPR.2019.00817 |
| [43] | Yang, F., Yang, H., Fu, J., et al. (2020) Learning Texture Trans-former Network for Image Super-Resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 14-19 June 2020, 5791-5800.
https://doi.org/10.1109/CVPR42600.2020.00583 |
| [44] | Jiang, Y., Chan, K.C., Wang, X., Loy, C.C. and Liu, Z. (2021) Robust Reference-Based Super-Resolution via C2-Matching. IEEE Conference on Computer Vision and Pattern Recognition, 19-25 June 2021, 2103-2112.
https://doi.org/10.1109/CVPR46437.2021.00214 |
| [45] | Cao, J., Liang, J., Zhang, K., et al. (2022) Reference-Based Image Super-Resolution with Deformable Attention Transformer. In: European Conference on Computer Vision, Springer Nature, Cham, 325-342.
https://doi.org/10.1007/978-3-031-19797-0_19 |
