Objective: To develop and evaluate an explainable framework for automated segmentation of atherosclerotic plaque morphological features in optical coherence tomography (OCT) images, integrating convolutional neural networks (CNNs) for image analysis and large language models (LLMs) for natural-language explainability. Background: Cardiovascular disease remains the leading cause of mortality worldwide, with atherosclerosis at its core. High-resolution imaging enables detailed visualization of plaque structures such as the lumen, fibrous cap, lipid core, and vasa vasorum. However, manual interpretation of OCT slices is time-consuming and requires expert knowledge. While CNN-based segmentation models can automate feature identification, they often lack interpretability. Existing explainability approaches, such as heat maps, highlight image regions used for prediction but fail to provide semantic reasoning useful to clinicians and researchers. This study explores the combined use of CNNs and LLMs (GPT, Claude, Gemini) to generate interpretable, human-readable explanations accompanying automated segmentation results. Methods: We utilized a curated OCT dataset of 25,698 annotated slices from 103 patients across two cardiovascular research centers. Each slice included manual annotations of key plaque morphological features (lumen, fibrous cap, lipid core, and vasa vasorum). A CNN-based segmentation model was trained using a 5-fold cross-validation design to ensure robust evaluation. LLMs were incorporated post hoc to interpret the CNN’s predictions, explaining morphological feature identification and potential clinical implications in natural language. The semantic quality, factual accuracy, and consistency of generated explanations across different LLMs were qualitatively assessed. Results: Quantitative and qualitative evaluation of the CNN segmentation performance and LLM-generated explanations is currently underway. Results, including segmentation accuracy, comparative model performance, and semantic analysis of LLM explanations, will be presented at the conference. Conclusions: This study introduces an explainable AI framework that unites CNN-based image segmentation with language-based reasoning for OCT plaque analysis. By complementing automated plaque identification with clear, contextually grounded explanations, this approach aims to enhance interpretability, reduce cognitive load for clinicians, and promote transparency in cardiovascular imaging research.
Gupta et al. (Tue,) studied this question.