基于增强特征对齐循环结构的视频超分辨
Video Super-Resolution Based on Enhanced Feature Alignment with Recurrent Structure

不同于单张图像超分辨，视频超分辨需要时空联合考虑以处理连续的低分辨视频从而获取清晰的高分辨连续帧序列。当前，随着深度学习在视频超分辨领域的广泛应用，深度视频超分辨虽已取得显著效果，但仍然存在视频特征信息挖掘不足的问题。特别是，从时间维度看，充分提取连续的视频时空特征并有效融合这些特征来实现帧内细腻、帧间稳定的视频超分辨，仍然是当前视频超分辨率研究的主要问题。在这项工作中，本文提出了一个基于增强特征对齐循环结构的视频超分辨网络。首先，我们通过多分支的特征提取模块从不同的深度对输入的特征进行信息提取。其次，在增强特征对齐循环结构中，本文提出从当前帧的多个方向上同时融合相邻帧的信息，并使用相应方向上的光流信息进行辅助对齐。最后，本文提出在多个上采样尺度上对超分结果进行增强。实验结果表明，所提出的方法能获得细节清晰、帧序列稳定的视频超分辨效果，在定量的评估指标和定性的可视化结果等方面都超越了近些年的其他先进方法。
Unlike single image super-resolution (SISR), video super-resolution (VSR) needs to be considered spatially and temporally to process continuous low-resolution video to obtain clear high-resolution continuous frames. At present, with the extensive application of deep learning in the field of video super-resolution, deep video super-resolution has achieved remarkable results, but there is still a problem of insufficient video feature information mining. In particular, from the perspective of time dimension, it is still the main problem of current video super-resolution research to fully extract continuous video spatio-temporal features and effectively integrate these features to achieve video super-resolution that is exquisite within frames and stable between frames. In this work, an enhanced feature alignment recurrent structure for video super-resolution network is proposed. Firstly, the network extracts information from the input features from different depths through a multi-branch feature extraction module. Secondly, in the enhanced deformable feature alignment recurrent structure, we propose to fuse the information of adjacent frames from multiple directions of the current frame at the same time, and use the optical flow information in the corresponding directions to assist alignment. Finally, we propose to enhance the SR results on multiple upsampling scales. The experimental results show that the proposed method can achieve video super-resolution effect with clear details and stable frame sequence, which surpasses other advanced methods in recent years in terms of quantitative evaluation and qualitative visualization results.