Abstract Segmentation and classification of lesion is essential for computer-aided diagnosis (CAD) in grayscale medical imaging, enabling precise localization and reliable categorization of various pathological conditions. However, conventional multi-task learning (MTL) frameworks often suffer from suboptimal task interactions and require manual loss weight tuning, limiting their effectiveness on challenging modalities such as ultrasound, MRI, and CT. To address these limitations, we propose a Dynamic Uncertainty-aware Network (DynU-Net), a multi-task framework that adaptively balances segmentation and classification through learnable per-task uncertainty parameters. This mechanism eliminates the need for manual hyperparameter tuning while mitigating gradient imbalance during optimization. Extensive experiments on three public datasets, including Brain Stroke CT, Brain Tumor MRI, and BUSI (ultrasound), demonstrate that DynU-Net consistently outperforms both single-task and existing multi-task baselines. In particular, it achieves a Dice score of 88.90% ± 0.33% and a macro-F1 score of 96.73% ± 0.74% on the Brain Stroke CT dataset while maintaining favorable computational efficiency compared to competing architectures. These results highlight the effectiveness of dynamic uncertainty-aware optimization in enabling robust and balanced multi-task learning for joint lesion segmentation and classification on public medical imaging benchmarks.
Tran et al. (Tue,) studied this question.