What question did this study set out to answer?

February 19, 2026npj Digital Medicine0 citationsOpen Access

Bridging radiology and pathology: domain-generalized cross-modal learning for clinical

XZXiang ZhongNanjing Agricultural University ZGZhuo GuSoochow University MSManimurugan ShanmuganathanUniversity of Tabuk

Key Points

To develop a unified framework for integrating mammography and histopathology for breast cancer diagnosis, addressing the limitations of traditional AI approaches.
Developed a shared vision transformer encoder with modality-specific adapters.
Implemented a weakly supervised contrastive alignment module for cross-modal learning.
Used domain generalization strategies like MixStyle augmentation and invariant risk minimization.
Applied causal test-time adaptation for handling unseen domains.
Achieved mean AUC of 0.90 under rigorous evaluation, outperforming existing models.
Reduced domain gaps significantly, demonstrating improved generalization across institutions.
Generated attention maps linking mammographic regions with histopathological evidence, enhancing interpretability.

Abstract

Reliable interpretation of clinical imaging requires integrating complementary evidence across modalities, yet most AI systems remain limited by single-modality analysis and poor generalization across institutions. We propose a unified cross-modal framework that bridges mammography and histopathology for breast cancer diagnosis through: (1) a shared vision transformer encoder with lightweight modality-specific adapters, (2) a weakly supervised patient-level contrastive alignment module that learns cross-modal correspondences without pixel-level supervision, (3) domain generalization strategies combining MixStyle augmentation and invariant risk minimization, and (4) causal test-time adaptation for unseen target domains. The model jointly addresses classification, lesion localization, and pathological grading while generating reasoning-guided attention maps that explicitly link suspicious mammographic regions with corresponding histopathological evidence. Evaluated on four public benchmarks (CBIS-DDSM, INbreast, BACH, CAMELYON16/17), the framework consistently outperforms state-of-the-art unimodal, multimodal, and domain generalization baselines, achieving mean AUC of 0.90 under rigorous leave-one-domain-out evaluation and substantially smaller domain gaps (0.03 vs. 0.06-0.10). Visualization and interpretability analyses further confirm that predictions align with clinically meaningful features, supporting transparency and trust. By advancing multimodal integration, cross-institutional robustness, and explainability, this study represents a step toward clinically deployable AI systems for diagnostic decision support.

Perguntar à IA

Bookmark

View Full Paper