Answers are grounded in this paper and cite their sources.
Vessel target recognition and localization are typically modeled using underwater acoustic signals, which contain a large amount of vessel operating characteristics and condition information. However, extracting operating characteristics from single signals faces heavy noise and non-stationarity challenges. Meanwhile, feature extraction using multimodal data faces the challenges of conflicting gradients between different modalities and ensuring the separability of vessel targets. To tackle these issues, we propose an audio-visual-textual features fusion method to recognize and localize vessel targets through Symbiotic Transformer (Symb-Trans) and Multi-View Regression (MVR) models. Specifically, the audio-visual samples are first preprocessed into paired time series and then projected into a unified optimization landscape via a Heterogeneous Batch Normalization (HetBN) layer to avoid gradient conflicts. Second, the Symb-Trans trains parallel encoders with cross-modal attention and embeds audio-visual representations for vessel target recognition. Finally, the MVR method learns neighboring target properties of a graph model from different perspectives, audio-visual-textual representations, to infer the collector-target distance. Since no off-the-shell multimodal dataset is available for vessel targets, we combine multiple public datasets, consisting of acoustic, and/or visual, and/or textural data, to obtain multimodal materials for model training and validation. Through experimental results and theoretical analysis, we show that Symb-Trans and MVR models outperform unimodal and generic multimodal state-of-the-art solutions for vessel target recognition and localization.
No takes yet. Share an insight, caveat, or question.
Liu et al. (2024) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: