Semi-automatic mask guidance enhances 3D tumor segmentation in medical imaging

Accurate tumor segmentation is essential for early diagnosis, treatment planning, and prognostic evaluation. Although manual annotation can achieve high accuracy, it is time-consuming and requires substantial expert involvement. While deep learning has significantly advanced medical image analysis, fully automated methods often fail to segment atypical lesions within complex abdominal anatomy, leading to missed lesions and misclassification of normal tissues, which may compromise clinical decision-making. To address these challenges, we incorporated guidance masks into a convolutional neural network (CNN)-based deep learning framework. Using our Star-Rain software, users place interactive clicks on lesion locations, and the system adaptively generates task-specific guidance masks. This approach directs the model’s attention to relevant regions, particularly in atypical or anatomically complex cases. Our method is validated on four independent cohorts comprising 1,217 CT scans from 726 patients, encompassing hepatic, renal, and pancreatic tumors. Across these datasets, our approach outperforms state-of-the-art baseline models on independent test sets, achieving Dice scores consistently above 0.7 and reducing the false negative rate (FNR) by 0.006 to 0.346 compared to the best fully automated approaches. In addition, the model’s segmentation outputs effectively support downstream prognosis tasks, highlighting its clinical value. These findings underscore the promise of semi-automatic deep learning frameworks that integrate minimal user input for reliable tumor segmentation. The proposed approach offers a practical and robust solution for clinical applications, enhancing segmentation accuracy and decision support while reducing the annotation burden. It is important to accurately determine the edge of tumors (tumor segmentation) prior to providing localized treatment. However, radiologists often find this process slow and labor-intensive. Fully automated computational methods can miss unusual or small lesions in the abdomen. To address this, we developed an interactive AI system combining clinical expertise and a computational approach known as deep learning. Unlike previous tools using simple points or boxes, our pipeline establishes guidance masks that capture the tumor’s shape. We showed it worked across four datasets covering liver, kidney, and pancreatic tumors. It was particularly successful at identifying small lesions often missed by automated models. Our approach could provide a high-precision solution for clinical use, improving diagnostic accuracy while significantly reducing the time needed for tumor segmentation by medical experts. Zhang et al. present an interactive deep learning system that uses guidance masks to segment abdominal tumors in CT images. Across four datasets covering liver, kidney and pancreatic tumors, the system improves segmentation, captures small lesions often missed by automated models.