Shot-Adaptive Video Super-Resolution through Reliability-Guided Multi-Level Refinement

The goal of an efficient video super-resolution techniques is to reduce computation by reusing data from previously processed frames in order to generate super-resolved videos. If an object is moving, is obstructed, or the scene changes, the features to be passed along may not be up-to-date. This paper introduces SARGM-VSR, a method called Shot-Adaptive Reliability-Guided Multi-Level Video Super-Resolution. The main concept is that of a reliability estimator that tests the reliability of the propagated features in regard to the improvement of the image quality. Based on these reliability scores SARGM-VSR controls the process at three levels: pixel, frame and shot. At the pixel level the system reduces the influence of features that're unreliable. It is at the frame level that it determines what proportion of the propagated state remains in use. The system also modifies the video refinement based on reliability. With features, a simple and fast branch takes care of the job. When reliability is low a more powerful refinement branch acts. This means that the system will achieve a balance between speed and accuracy. Overall SARGM-VSR improves reconstruction quality while keeping costs manageable. Experiments on the REDS4 benchmark show that SARGM-VSR achieves a PSNR of 32.90 dB and an SSIM of 0.910, improving upon BasicVSR++ by 0.80 dB and 0.008, respectively. Further evaluation on the full REDS test set shows improvements in perceptual and temporal quality. Compared with BasicVSR++, LPIPS decreases from 0.132 to 0.122 (7.58%), flow-based temporal error (tOF) from 0.249 to 0.215 (13.65%), and warping error, E_warpfrom 〖2.2x10〗^(-3) to 〖2.12x10〗^(-3) that is by 5.19%. Ablation studies highlight the contribution of the individual components. The learned reliability maps also indicate when propagated features are likely to benefit reconstruction.