ScholarWorks@숙명여자대학교

초록

Deep learning-based video super-resolution (VSR) is a crucial technology for enhancing video frame quality, relying on the effective utilization of spatial and temporal correlations. VSR is widely applied across various real-world scenarios, including video surveillance systems, medical and satellite image reconstruction, and video streaming services. Recent research predominantly focuses on residual block-based and transformer based backbones demonstrating notable effectiveness. However, many methods treat spatial features uniformly, resulting in inadequate information acquisition and detail enhancement. This paper proposes the hierarchical recurrent transformer (HiRT) to enhance the recurrent propagation in the frequency domain. The proposed propagation consists of uni-directional backward and forward stages, as well as a bi-directional stage. This structure can deal with various types of motion. The proposed HiRT comprises three transformer modules. The global transformer block improves low-frequency features. The high-frequency components are enhanced in the local transformer. Alongside the image transformer, incorporating discrete wavelet transform (DWT)based transformer processes can enhance both background and edge details. The proposed HiRT outperforms all compared state-of-the-art (SOTA) methods in terms of structural similarity index measure (SSIM) on the realistic and dynamic scenes 4 (REDS4) and the video sequences 4 (Vid4) benchmarks. Especially, the proposed HiRT surpasses the video restoration transformer (VRT) which is the transformer-based SOTA method with 0.32 decibel (dB) and 0.0068 on REDS4 in peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM), respectively. The proposed HiRT can brings about 0.12 dB and 0.07 dB higher PSNR performances compared to the basic video super-resolution++ (BasicVSR++) on REDS4 and Vid4, respectively.

키워드