Multi-Scale Residual Image Stitching Network Combining Homography and Thin-Plate Spline Motion

Image stitching is a key technology in computer vision, whose purpose is to generate a seamless high-definition image from multiple images. Traditional image stitching algorithms rely too much on feature detection quality and suffer from poor feature extraction ability, artifacts, misalignment and distortion in scenes such as of low texture. Deep learning based stitching networks have better feature detection ability with strong robustness, but most of which focus on estimating a single global homography, therefore the images cannot be fully aligned in scenes with large parallax. To solve these problems, this paper proposes a new unsupervised multi-scale residual stitching network, which stacks the input image feature maps at different scales to form a feature pyramid, and incorporates a shared weight Convolutional Block Attention Module (CBAM) into each pyramid layer. Through joint optimization of homography and thin-plate spline motion, a flexible warping from global to local is implemented, which could process images with large parallax and large baseline. Experiments show that the objective performance of this method is better than existing algorithms, and the image registration is more accurate in challenging scenes.