Symmetry-Driven Multimodal Adversarial Attacks: An Information-Theoretic Perspective on Cross-Modal Invariance and Robustness

Multimodal models such as CLIP and ALBEF essentially maximize cross-modal mutual information to align heterogeneous modalities, utilizing semantic consistency as an implicit prior. However, this alignment mechanism creates a structural vulnerability: the models rely heavily on invariant information coupling. In this work, we investigate this vulnerability and propose a symmetry-driven adversarial attack framework. Unlike standard methods that inject high-entropy unstructured noise, our approach designs collaborative perturbations by modeling semantic-consistent mappings between geometric image transformations and syntactic text variations. By explicitly exploiting the information redundancy inherent in cross-modal symmetries, our method effectively reduces the entropy of the adversarial search space. This reveals a fundamental trade-off between information invariance and robustness, achieving state-of-the-art attack success rates with imperceptible perturbations.