CancerFusionPrompt: A Novel Framework for Multimodal Cancer Subtype Classification Using Vision‐Language Model

This is the authors' abstract. We don't add key points for this paper.

Background Cancer subtype classification plays a pivotal role in personalised medicine, requiring the integration of diverse data types. Traditional prompting methods in vision‐language models fail to fully leverage multimodal data, particularly when working with minimal labelled data. Methods To address these limitations, we propose a novel framework that introduces the CancerFusionPrompt, a specialised prompting method for integrating imaging and multi‐omics data. Our proposed approach extends the few‐shot learning paradigm by incorporating in‐context learning for cancer subtype classification. Results The proposed method significantly outperforms state‐of‐the‐art techniques in cancer subtype classification, achieving notable improvements in both accuracy and generalisation. These results demonstrate the superior capability of CancerFusionPrompt in handling complex multimodal inputs compared to existing prompting methods. Conclusions The CancerFusionPrompt framework offers a powerful solution for integrating multimodal data in cancer subtype classification tasks. By overcoming the limitations of current prompting methods, CancerFusionPrompt approach enables more accurate and robust predictions with minimal labelled data.