Adaptive Prompt Evolution for Continual Learning in Diabetic Retinopathy Detection

Abstract We introduce Adaptive Prompt Evolution (APE), a framework that autonomously optimizes text prompts for continual learning (CL) in medical imaging applications. Building upon task-agnostic domain-incremental learning, APE addresses static prompt engineering limitations through intelligent feedback loops that evolve templates and medical descriptions using iterative performance assessment. Our approach replaces manual prompt crafting with a local medical language model (Med-Gemma 4B) that generates and refines prompt candidates based on zero-shot clustering performance, maintaining complete offline operation for healthcare compliance. APE demonstrates strong efficiency, requiring only 50 evaluations to achieve F1-score of 0. 9229, compared to exhaustive search requiring 110 evaluations for F1-score of 0. 8921, representing 2. 2 computational efficiency with 3. 45% performance improvement. Evaluated on diabetic retinopathy detection across three increasingly complex tasks, we integrate APE with established CL strategies including Elastic Weight Consolidation (EWC), Gradient Episodic Memory (GEM), and Learning Without Forgetting (LwF). On experiments across Multi-Layer Perceptron (MLP), Residual, and Attention architectures, APE outperforms baselines across all strategies. The framework reduces privacy exposure during experience replay by storing only embeddings rather than raw images, while demonstrating robust generalization performance across tasks with minimal degradation within the studied diabetic retinopathy setting.