What question did this study set out to answer?

The research aims to develop a novel framework for detecting anomalies in ultra-widefield fundus images using CLIP-based methods.

April 12, 2026Open Access

Parameter-Efficient Contrastive Language-Image Pre-training (CLIP) Adaptation for Few-Shot Anomaly Detection in Ultra-Widefield Fundus Images

Key Points

The research aims to develop a novel framework for detecting anomalies in ultra-widefield fundus images using CLIP-based methods.
Developed a dual-pathway mechanism for precise localization of anomalies.
Implemented a lightweight low-rank adapter for efficient fine-tuning.
Created a Poisson-blending synthesis technique for generating abnormal training samples.
Evaluated the framework across 8 datasets with only 32 training pairs.
Achieved an average AUC of 85.23% and a peak AUC of 93.43% across evaluations.
Demonstrated robust performance on a dataset of 29,739 images with over 50 anomaly patterns.
Provided an efficient solution with low computational costs and data requirements.

Abstract

• A huge dataset including 29,739 images covering more than 50 anomaly patterns • First CLIP-based anomaly detection framework for ultra-widefield fundus images • Dual-pathway mechanism enables precise localization of diverse ocular anomalies • Lightweight low-rank adapter achieves domain adaptation with minimal parameters • Poisson-blending synthesis overcomes variety and scarcity of abnormal training samples • Robust performance across 8 datasets using only 32 training pairs (Average AUC: 85.23%) Fundus diseases are among the leading causes of visual impairment and blindness, many of which could be prevented through early intervention. Ultra-widefield fundus (UWF) imaging, offering a wide field of view and enabling non-mydriatic acquisition, has emerged as an ideal modality for screening; however, the volume of cases far exceeds the diagnostic capacity of ophthalmologists, necessitating an automated diagnostic system. Nevertheless, existing deep learning algorithms often encounter challenges including high computational costs, large data requirements, and performance degradation on out-of-distribution data. Domain-adaptive fine-tuning of existing multimodal large models offers a promising pathway to address these limitations. Here, we developed an anomaly detection framework for UWF images based on CLIP. First, we proposed a dual-pathway detection mechanism for accurate anomaly detection. Second, a lightweight hybrid low-rank adapter is designed for parameter-efficient fine-tuning to bridge the domain gap between natural images and fundus images. Moreover, to address the scarcity and variability of abnormal samples, we developed an anomaly synthesis algorithm based on Poisson-blending to broaden and generalize the anomaly paradigm. With only 32 pairs of training samples, the framework achieved efficient and robust anomaly detection. The framework was evaluated on 8 independent cross-source datasets (29,739 images covering more than 50 anomaly patterns), yielding an average AUC of 85.23% and a peak AUC of 93.43%. Owing to its low computational and data requirements, this framework provides a practical and scalable solution for fundus disease screening and shows strong potential for clinical translation. Code and details are available through https://github.com/JohnLeo-XJTU/AnomalyUWF .

Read Full Paperexternally

AI से पूछें

Bookmark

View Full Paper