|
|
基于Transformer和不可分加性小波的图像超分辨率重建
|
Abstract:
针对目前超分辨率重建存在纹理模糊、扭曲等问题，提出了一种基于Transformer和不可分加性小波的网络。该网络由小波分解模块、纹理提取模块、浅层特征提取模块、用于纹理匹配的相关嵌入模块、纹理传输模块、用于纹理融合的跨尺度集成模块共六个模块组成。我们对此网络的重建性能在相关的测试数据上与已有的典型方法进行了对比研究，实验结果表明，该网络不仅提升了视觉效果，而且获得了较好的客观指标评价，在所比较的六种方法中，所构建网络在CUFED5、Sun80和Manga109数据集上的峰值信噪比(PSNR)和结构相似度(SSIM)都具有最佳性能。
Aiming at the problems of texture blur and distortion in super-resolution reconstruction, a network based on Transformer and non-separable additive wavelet is proposed. The network consists of six modules: Wavelet Decomposition module, Texture Extraction module, Shallow Feature Extraction module, Relevance Embedding module for texture matching, Texture Transmission module, and Cross Scale Integration module for texture fusion. We compared the reconstruction performance of this network with the existing typical methods on the relevant test data. The experimental results show that this network not only improves the visual effect, but also obtains better objective index evaluation. Among the six methods compared, the peak signal to noise ratio (PSNR) and structure similarity (SSIM) of the constructed network on CUFED5, Sun80 and Manga109 datasets have the best performance.
| [1] | Dong, C., Loy, C.C., He, K., et al. (2014) Learning a Deep Convolutional Network for Image Super-Resolution. European Conference on Computer Vision, Zurich, 6-12 September 2014, 184-199.
https://doi.org/10.1007/978-3-319-10593-2_13 |
| [2] | Dong, C., Loy, C.C. and Tang, X. (2016) Accelerating the Super-Resolution Convolutional Neural Network. European Conference on Computer Vision, Amsterdam, 11-14 October 2016, 391-407.
https://doi.org/10.1007/978-3-319-46475-6_25 |
| [3] | Kim, J., Lee, J.K. and Lee, K.M. (2016) Accurate Image Super-Resolution Using Very Deep Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 1646- 1654. https://doi.org/10.1109/CVPR.2016.182 |
| [4] | Kim, J., Lee, J.K. and Lee, K.M. (2016) Deeply-Recursive Convolutional Network for Image Super-Resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 1637- 1645. https://doi.org/10.1109/CVPR.2016.181 |
| [5] | Ledig, C., Theis, L., Huszár, F., et al. (2017) Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 21-26 July 2017, 4681-4690. https://doi.org/10.1109/CVPR.2017.19 |
| [6] | Johnson, J., Alahi, A., et al. (2016) Perceptual Losses for Real-Time Style Transfer and Super-Resolution. European Conference on Computer Vision, Amsterdam, 11-14 October 2016, 694-711.
https://doi.org/10.1007/978-3-319-46475-6_43 |
| [7] | Lim, B., Son, S., Kim, H., et al. (2017) Enhanced Deep Residual Networks for Single Image Super-Resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, 21-26 July 2017, 136-144. https://doi.org/10.1109/CVPRW.2017.151 |
| [8] | Wang, X., Yu, K., Wu, S., et al. (2018) Esrgan: Enhanced Super-Resolution Generative Adversarial Networks. Proceedings of the European Conference on Computer Vision (ECCV) Workshops, Munich, 8-14 September 2018, 63-79.
https://doi.org/10.1007/978-3-030-11021-5_5 |
| [9] | Zheng, H., Ji, M., Han, L., et al. (2017) Learning Cross-Scale Correspondence and Patch-Based Synthesis for Reference-Based Super-Resolution. Proceedings of the British Machine Vision Conference, London, 4-7 September 2017, Article No. 138. https://doi.org/10.5244/C.31.138 |
| [10] | Zheng, H., Ji, M., Wang, H., et al. (2018) Crossnet: An End-to-End Reference-Based Super Resolution Network Using Cross-Scale Warping. Proceedings of the European Conference on Computer Vision (ECCV), Munich, 8-14 September 2018, 88-104. https://doi.org/10.1007/978-3-030-01231-1_6 |
| [11] | Zhang, Z., Wang, Z., Lin, Z., et al. (2019) Image Super-Resolution by Neural Texture Transfer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, 15-20 June 2019, 7982-7991.
https://doi.org/10.1109/CVPR.2019.00817 |
| [12] | Yang, F., Yang, H., Fu, J., et al. (2020) Learning Texture Transformer Network for Image Super-Resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 13-19 June 2020, 5791-5800.
https://doi.org/10.1109/CVPR42600.2020.00583 |
| [13] | Vaswani, A., Shazeer, N., Parmar, N., et al. (2017) Attention Is All You Need. 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, 4-9 December 2017, 30. |
| [14] | Chen, Q.H., Micchelli, C.A., Peng, S.L., et al. (2003) Multivariate Filter Banks Having Matrix Factorizations. SIAM Journal on Matrix Analysis and Applications, 25, 517-531. |
| [15] | 刘斌, 彭嘉雄. 基于二通道不可分加性小波的多光谱图像融合[J]. 光学学报, 2007(8): 1419-1424. |
| [16] | Nunez, J., Otazu, X., Fors, O., et al. (1999) Multiresolution-Based Image Fusion with Additive Wavelet Decomposition. IEEE Transactions on Geoscience and Remote Sensing, 37, 1204-1211. https://doi.org/10.1109/36.763274 |
| [17] | Simonyan, K. and Zisserman, A. (2014) Very Deep Convolutional Networks for Large-Scale Image Recognition. 3rd International Conference on Learning Representations (ICLR 2015), San Diego, 7-9 May 2015, 1-14. |
| [18] | Woo, S., Park, J., Lee, J.Y., et al. (2018) Cbam: Convolutional Block Attention Module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, 8-14 September 2018, 3-19.
https://doi.org/10.1007/978-3-030-01234-2_1 |
| [19] | Shi, W., Caballero, J., Huszár, F., et al. (2016) Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 1874-1883. https://doi.org/10.1109/CVPR.2016.207 |
| [20] | Kingma, D.P. and Ba, J. (2014) Adam: A Method for Stochastic Optimization. arXiv: 1412.6980. |
