Differential Reinforcement and Global Collaboration Network for RGBT Tracking

RGB-thermal (RGBT) tracking has received extensive attention as a research hotspot, benefiting from the complementarity of visible (RGB) and thermal (T) modalities, which can better handle tracking tasks in harsh environments. Existing studies typically deploy the same structure to extract features of heterogeneous modalities with different properties, which may be difficult to simultaneously search for mining strategies suitable for each modality, resulting in insufficient feature representation. With the success of transformer, how to jointly consider the heterogeneity and global mining to facilitate better integration of heterogeneous modalities is also a crucial issue worth exploring. To solve these problems, we propose a unique differential reinforcement and global collaboration network for RGBT tracking, which includes a parallel backbone, modal differential reinforcement (MDR), and global mining collaboration module (GMCM). Specifically, MDR introduces raw data to control adaptive differential reinforcement for both the modalities via a modality-sharing augmentation unit and a modality-aware gating unit. The data-driven design makes it easier for the two modalities to search for the respective appropriate mining strategies, further effectively enhancing the respective feature representations. GMCM first mines intra-modal and cross-modal global information separately, and then performs an adaptive ensemble operation to balance the two. This operation learns intra-modal and cross-modal global weights in RGB and T modalities to better integrate heterogeneous modalities based on heterogeneity and global cues. Extensive experiments on three datasets demonstrate that the performance of the proposed network is comparable to state-of-the-art RGBT trackers.