A unified attention-gated multi-task network for joint classification and lesion segmentation of breast ultrasound images

Breast ultrasound (BUS) is widely used for breast cancer screening, but its interpretation is operator-dependent. In computer-aided diagnosis, lesion classification and lesion segmentation are conventionally addressed by separate deep-learning models, which increases the parameter budget, computational cost, and complexity of clinical deployment. It remains unclear whether a single multi-task model can match the performance of two dedicated single-task models while reducing this overhead. We developed an attention-gated multi-task learning (MTL) framework built on a shared ResNet-34 encoder that simultaneously performs three-class classification (normal, benign, malignant) and lesion segmentation in a single forward pass. Using the publicly available BUSI dataset (780 images; stratified 70%/15%/15% train/validation/test split; fixed random seed for reproducibility), we compared the unified model against three dedicated baselines: a classification-only model, a segmentation-only model, and a segmentation-only model equipped with attention gates. Classification models were compared using the paired McNemar test, segmentation models using the paired Wilcoxon signed-rank test, and bootstrap 95% confidence intervals (1,000 resamples) were computed for the principal metrics. Model efficiency was quantified by parameter count. On the held-out test set ( n = 117), the unified MTL model achieved a classification accuracy of 93.16%, macro F1-score of 0.931, and macro AUC of 0.984, statistically comparable to the dedicated classification model (accuracy 91.45%; McNemar p = 0.63). Its lesion segmentation performance (mean Dice 0.766, 95% CI 0.710–0.819) was statistically comparable to the dedicated segmentation model (Dice 0.810; Wilcoxon p = 0.22). Adding attention gates to the segmentation-only baseline did not significantly change segmentation accuracy (Dice 0.813 vs. 0.810; p = 0.50). The unified model preserved clinically important behaviour, with 100% recall for normal images and no malignant case misclassified as normal. Critically, the unified model required 24.7 million parameters, compared with 45.9 million for two separate task-specific models—a 46% reduction. A single attention-gated multi-task model can deliver breast ultrasound classification and lesion segmentation with accuracy statistically comparable to that of dedicated single-task models, while nearly halving the parameter count. These findings support the value of the unified framework for efficient, integrated computer-aided diagnosis and workflow consolidation, rather than for accuracy gains over single-task approaches.