A multimodal clinical decision support system for retinal diseases detection and personalized disease progression and severity analysis

Visual impairment affects 2.2 billion people globally, with over 1 billion cases preventable through early intervention. Retinal diseases including choroidal neovascularization, diabetic macular edema, and drusen constitute leading causes of visual impairment. Clinical Decision Support Systems (CDSSs) have potential to enhance retinal diagnosis from optical coherence tomography (OCT) scans. However, current CDSSs face three critical limitations. Firstly, high-performing CDSSs require extensive preprocessing and fail with speckle, noise-like pattern formed from light scattering off retinal microstructures. Secondly, current CDSSs do not quantify disease severity and finally the lack clinical interpretability, preventing clinical adoption. A multimodal CDSS could be developed addressing each limitation. The Speckle-Aware Dynamic Vision Transformer (SA-DVT) framework leverages speckle as a diagnostic cue for feature extraction from OCT images. The Severity Estimation and Personalized Analysis (SEPIA) framework provided severity estimation through computing Euclidean distance from healthy anatomy. The Clinical Reasoning and Analysis Framework for Trust utilizes t-SNE dimensionality reduction, Swin-UNet segmentation, and GPT-5 patient report generation to create interpretable outputs across graphical, visual, and textual modalities. SA-DVT achieved 96.9% disease classification accuracy with precision and recall exceeding 96% across disease categories. SEPIA achieved 96.6% bootstrap confidence for longitudinal tracking. Dimensionality reduction extracted SA-DVT and SEPIA feature vectors in 2D space for analyzing disease classification patterns. Retinal layer segmentation achieved 0.9393 Dice coefficient, revealing biomarkers and layer deformations. Patient reports with retinal status, biomarkers, and clinical summaries demonstrated 100% consistency across five independent runs. This CDSS has potential as a clinical tool for retinal disease management.