Clinical Knowledge-Guided PET/CT Lesion Segmentation with Interpretable Fusion of Metabolic and Structural Cues

Automated whole-body lesion segmentation in 18F-FDG PET/CT images marks a pivotal breakthrough in oncological diagnostics, substantially improving the accuracy and efficiency of tumor burden assessment. Manual segmentation is often plagued by significant interobserver variability, underscoring the necessity for automated solutions. The synergistic combination of PET's exceptional sensitivity for detecting metabolic activity with CT's anatomical precision renders accurate segmentation crucial for achieving quantitative and reproducible clinical workflows. However, current methodologies frequently grapple with challenges such as over-segmentation or under-segmentation, inadvertently delineating normal tissues with elevated uptake or neglecting lesions characterized by subtle intensity variations, primarily due to a lack of integrated metabolic and anatomical insights. To address these limitations, we present a novel framework that adeptly integrates clinical expertise regarding anatomical and metabolic cues to refine PET/CT lesion segmentation. Our innovative mixture-of-experts (MoE) based interpretable fusion module skillfully merges complementary modality information while explicitly elucidating the pixel-level contributions of each modality to the final segmentation outcome. Rigorous evaluations across three in-domain benchmarks and two external datasets demonstrate our model's superior segmentation performance and generalizability. Furthermore, our visualizations provide compelling insights into the pivotal role each modality plays in the decision-making process, highlighting our approach's transformative potential in enhancing PET/CT lesion segmentation. Building on this foundation, we further validated the prognostic significance of the features extracted from our proposed framework in the context of PET/CT-based prognosis predictions.