Answers are grounded in this paper and cite their sources.
Situation awareness (SA) in disaster management requires robust modeling of multimodal big data from complex environments. With the increasing availability of large-scale video and audio data from surveillance systems, drones, and public sensors, multimodal approaches have become essential for improving reliability. However, existing methods often rely on uniform data sampling and treat modalities equally, limiting their effectiveness when data quality varies across time and modality. Compared with our previous work, this study extends from modality-specific analysis to a multimodal audio–visual framework. To address these challenges, this paper proposes MSA-SD², a multimodal situation awareness framework specifically designed for big data environments based on selective dual self-distillation. An adaptive temporal sampling and alignment module is introduced to prioritize informative temporal segments and ensure consistent audio–visual correspondence, enabling efficient processing of large-scale multimodal big data. For visual representation, a superpixels-guided vision transformer is employed, while a transformer-based audio encoder models temporal acoustic patterns. A multimodal student model is guided by visual and audio teacher models through a selective distillation strategy that adapts to modality reliability. Experimental results demonstrate improved performance on large-scale multimodal datasets, particularly under challenging big data conditions with varying data quality across modalities.
No takes yet. Share an insight, caveat, or question.
Chen et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: