Visible-infrared Dual-sensor Tracking based on Transformer via progressive feature enhancement and fusion

This paper investigates how to implement accurate RGB-T tracking by achieving effective feature enhancement of the target and adaptive fusion of the complementary information in RGB and thermal infrared modalities. Inspired by the excellent long-range dependency modeling ability of transformer, we propose a novel RGBT tracking method based on transformer via progressive feature enhancement and fusion. The overall flowchart of our proposed tracker consists of a two-branch Siamese network, respectively an exemplar branch and a search branch. Firstly, deep features of the RGB and thermal infrared images are extracted by a backbone. And then the features in each branch are enhanced progressively in the channel and spatial dimensions. Specifically, in the channel dimension, channel attention feature module (CAFM) is designed to adaptively enhance the RGB and thermal infrared features. And in the spatial dimension, transformer self-attention mechanism with AiA module is integrated to enhance the dual-modality features. Next, the enhanced features from the exemplar and search branches are fused based on the transformer cross-attention mechanism, which can achieve global and deep interaction between the exemplar and search images. Finally, the fused features are fed into a corner predictor head to estimate the target state. Experiments on two widely used public benchmarks (RGBT234 and LasHeR) demonstrate the effectiveness and efficiency of our proposed method when compared to many other state-of-the-art (SOTA) trackers released recently.