Multimodal evidence is critical in computational pathology: gigapixel whole slide images capture tumor morphology, while patient-level clinical descriptors preserve complementary context for prognosis. Integrating such heterogeneous signals remains challenging because feature spaces exhibit distinct statistics and scales. We introduce MMSF, a multitask and multimodal supervised framework built on a linear-complexity MIL backbone that explicitly decomposes and fuses cross-modal information. MMSF comprises a graph feature extraction module embedding tissue topology at the patch level, a clinical data embedding module standardizing patient attributes, a feature fusion module aligning modality-shared and modality-specific representations, and a Mamba-based MIL encoder with multitask prediction heads. Experiments on CAMELYON16 demonstrate state-of-the-art performance with up to 4.7% accuracy improvement over the best existing method, while achieving the highest accuracy and competitive AUC on TCGA-NSCLC. Evaluations on five TCGA survival cohorts yield 7.1–9.8% C-index improvements compared with unimodal methods and 5.6–7.1% over multimodal alternatives. The project source code is publicly available at https://github.com/chengyingshe/MMSF .
She et al. (Tue,) studied this question.