Liver-VLM: Enhancing Focal Liver Lesion Classification with Self-Supervised Vision-Language Pretraining

Accurate classification of focal liver lesions (FLLs) is crucial for reliable clinical decision-making. Inspired by contrastive vision-language models such as CLIP and MedCLIP, we propose Liver-VLM for FLLs classification, trained on a dedicated multi-phase 2D CT dataset. Liver-VLM aligns multi-phase CT image representations with class-specific textual descriptions by calculating their similarity under a cross-entropy loss. Furthermore, we design tailored, enriched textual prompts to stabilize optimization and enable robust classification even with limited labeled data. Additionally, self-supervised pretraining and data augmentation strategies are incorporated to further improve classification performance. Experimental results on an in-house MPCT-FLLs dataset demonstrate that Liver-VLM consistently outperforms existing VLMs, achieving an accuracy of 85.63 ± 3.18% and an AUC of 0.94 ± 0.01. Our findings highlight the efficacy of self-supervised learning and task-specific augmentation in overcoming data scarcity and distributional biases in medical image analysis.