Adaptive cross-domain multimodal few-shot learning with hierarchical vision language alignment

Multimodal foundation models have achieved remarkable success in vision–language understanding; however, their performance deteriorates substantially in cross-domain few-shot learning scenarios due to three interrelated challenges that are particularly acute when target domains differ drastically from pretraining data: limited labeled data that prevents adequate fine-tuning and amplifies overfitting risk, significant domain shifts that undermine learned representations because source–target distribution mismatch invalidates pretrained feature statistics, and inadequate fine-grained vision–language alignment where global contrastive objectives fail to capture local patch-level correspondences critical for specialized recognition. To address these challenges, this paper proposes Adaptive Multimodal Few-Shot Learning with Cross-Domain Knowledge Transfer (AMFS-CDT), a unified framework designed to enhance robust multimodal representation learning across diverse domains. The framework incorporates a patch-level generative augmentation mechanism that synthesizes semantically consistent local visual features to mitigate data scarcity, a hierarchical vision–language alignment strategy that leverages patch-to-text contrastive learning and cycle-consistency constraints to strengthen fine-grained cross-modal correspondence, and a domain-adaptive meta-learning mechanism that enables rapid knowledge transfer while preserving domain-invariant semantic representations under distribution shifts. Extensive experiments on five benchmark datasets covering medical imaging, remote sensing, and fine-grained visual recognition demonstrate that the proposed framework consistently surpasses existing state-of-the-art multimodal few-shot learning methods, achieving classification accuracy improvements ranging from 8.7% to 15.3% in challenging cross-domain settings. Comprehensive ablation studies verify the contribution of each component, while attention-based visualizations demonstrate improved interpretability by highlighting meaningful vision–language correspondences. In addition, theoretical generalization analysis provides insights into the robustness of the proposed approach under domain adaptation. Overall, AMFS-CDT offers an effective, interpretable, and scalable solution for cross-domain multimodal few-shot learning, making it particularly suitable for resource-constrained domain-specific applications where annotated data are scarce.