Segmentation-Prior Guided Multimodal Learning for Breast Ultrasound Diagnosis

Deep learning shows promise for breast ultrasound segmentation and classification, yet modality heterogeneity and scarce annotations often limit performance. We propose Segmentation-Prior Guided Multimodal Learning for Breast Ultrasound Diagnosis, a unified framework for joint lesion segmentation and malignancy classification. An optimized SAM-based SAMUS generates multimodal lesion masks as spatial priors. Guided by these masks, GAB extracts lesion-specific features, while a hybrid fusion module integrates cross-modal multi-head attention for semantic alignment with a dynamic weight generator for sample- adaptive modality weighting. Experiments demonstrate consistent gains, achieving Dice scores of 82.13%/80.98%/73.84% (B-mode/SE/CEUS) and 93.62% classification accuracy.