CASS: A Context-Aware Memory Framework With Spectral–Spatial Modulation for UAV Multispectral Video Tracking

Unmanned Aerial Vehicle (UAV) multispectral video object tracking is critical for real-world applications. While multispectral imaging offers complementary spectral cues beyond the visible range, tracking in aerial scenarios remains challenging due to data scarcity, suboptimal spectral-spatial modulation, and discrete sequential temporal modeling. To this end, we propose CASS, a context-aware memory framework with spectral-spatial modulation, which integrates spectral, spatial, and temporal cues for UAV multispectral tracking. CASS introduces two lightweight modules: 1) the efficient spectral-spatial modulation (ESSM) module, which modulates spatial representations through spectral-guided fusion, and 2) the context state space reasoning (CSSR) module, which leverages evolving state space memory to retain long-term temporal cues and mitigate error propagation during cross-frame reasoning. By integrating these components in a parameter-efficient fine-tuning fashion, CASS achieves both efficient modulation and context-aware tracking. Evaluations on the UAV multispectral benchmark (MUST) and ground-based hyperspectral benchmarks (NIR, RedNIR, VIS, MSSOT, MSVT) demonstrate CASS’s superior performance for both general hyperspectral tracking and specific UAV perception.