Answers are grounded in this paper and cite their sources.
Experimental study reveals enhanced reasoning accuracy and noise robustness in multimodal language models, suggesting cross-modal routing improves inference without external data.
Current augmentation of Large Language Models (LLMs) primarily relies on external corpora, which constrains performance based on database availability and leaves models vulnerable to input perturbations. Inspired by the human perceptual system’s ability to leverage mutual augmentation across different modalities, We propose Intrinsic Modality Disparity Augmentation (IMD), a reliability-aware cross-modal re-encoding framework that does not rely on external retrieval databases during inference, while using a labeled calibration set to estimate modality-specific routing centroids. Our approach consists of two core components: first, an on-demand synthesis mechanism that assesses the reliability of the initial query and generates complementary modal representations when text is deemed insufficient; second, a dynamic cross-modal latent space routing module that evaluates and ranks candidates based on their proximity to learned expert centroids. Extensive experiments on diverse reasoning benchmarks demonstrate that IMD significantly outperforms the MLLM’s native text-only inference and standard augmentation methods, delivering substantial gains in both reasoning accuracy and robustness against textual noise. Furthermore, efficiency analysis reveals that IMD maintains a favorable balance between performance and computational overhead, achieving 3x speedups over naive multimodal conversion and remaining highly competitive with traditional text-based augmentation methods.
No takes yet. Share an insight, caveat, or question.
Wu et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: