Dynamic Feature-Memory Transformer Network for RGBT Tracking

RGBT object tracking is an important research topic due to the complementary features of visible and thermal infrared images. However, the established fast-tracking Siamese series RGBT trackers do not fully exploit the predicted results and global information, which leads to poor tracking performance in challenging scenarios such as target object deformation and fast movement. To address these challenges, we propose a dynamic feature-memory transformer RGBT tracking framework in this article. Precisely, we extract the features of visible and infrared image pairs separately using convolutional networks. We then fuse the complementary semantic information using our proposed complementary semantic fusion module, which consists of a group convolution, a channel attention structure, a flattening operation, and channel reduction. Next, we concatenate the dynamic memory feature extracted from the predicted frame pair, the template pair feature, and the search region pair feature. Then input concatenated features into transformer encoder-decoder modules to acquire the global information and further fuse the RGBT features. Finally, we predict the object state using the tracking head while deciding whether to update the dynamic memory feature using the reliability estimator. We conducted extensive experiments on the widely used RGBT234 and LasHeR datasets to evaluate the proposed framework’s performance. The experimental results show that our method outperforms state-of-the-art RGBT trackers, demonstrating the effectiveness of our approach. Overall, our proposed framework leverages visible and infrared images and incorporates global information and dynamic memory features, offering a promising solution to the challenges in RGBT object tracking. Code will be open sourced at https://github.com/ELOESZHANG/DFMTNet