Real‐World External Validation of Artificial Intelligence‐Based Full‐Vessel Segmentation for Intracoronary Optical Coherence Tomography

Q: What is the clinical evidence from this study?

Study design: Observational. Population: Coronary artery disease undergoing clinically indicated intracoronary optical coherence tomography (OCT) (n=100). Intervention: AI-based automated OCT full-vessel segmentation algorithm (OCT-AID) vs. Manual OCT image analysis by expert readers (consensus reference standard and interobserver variability). Primary outcome: Frame-level agreement for calcified plaque identification between AI algorithm and expert consensus vs interobserver agreement (Kappa 0.88 vs 0.90, 95% CI 0.84-0.92, p=0.022).

Q: What does this research mean for the field?

The OCT-AID artificial intelligence algorithm accurately performs automated full-vessel segmentation in real-world intracoronary OCT, achieving excellent agreement with expert consensus for calcified plaque and exceeding interobserver agreement for lipid plaque. Novelty: ClaimNovelty.CONFIRMATORY. Consensus alignment: ConsensusAlignment.NEUTRAL.

Rick H. J. A. Volleberg; Doosup Shin; Ruben G.A. van der Waerden; Craig R. Porter; Susan V. Thomas; Fernando Sosa; Simone Saitta; Aysun Cetinyurek‐Yavuz; Joske L. van der Zande; Thijs J. Luttikholt; Pierandrea Cancian; Xiaojin Gu; Leah Heil; Jos Thannhauser; Clara I. Sánchez; Bram van Ginneken; Ivana Išgum; Allen Jeremias; Evan S. Shlofmitz; Richard Shlofmitz; Ziad A. Ali; Niels van Royen

doi:10.1161/jaha.125.049353

Key result

OCT-AID automated segmentation matches expert consensus on calcified plaque and outperforms interobserver lipid detection.

Kappa 0.88 vs 0.90
95% CI 0.84-0.92
P=0.022
n=100

Why the study?

Algorithms for automated evaluation of intracoronary OCT images are mostly developed and validated on well-curated data sets that may not represent real-world data, so external validation in an unselected consecutive real-world data set was needed.

Does the OCT-AID artificial intelligence algorithm accurately perform automated full-vessel segmentation and plaque quantification compared to manual expert consensus in a real-world cohort of patients undergoing clinically indicated intracoronary OCT?

Key Points

To externally validate the diagnostic performance of a deep learning algorithm for full-vessel segmentation of intracoronary optical coherence tomography images in an unselected real-world patient cohort.
Retrospective, single-center external validation study evaluating 100 consecutive patients undergoing clinically indicated intracoronary optical coherence tomography.
Applied the OCT-AID deep learning model for automated pixelwise labeling across 2,560 analyzable frames, comparing AI predictions against manual expert annotations as the reference standard.
AI agreement with manual reference was excellent for calcified plaque identification (κ=0.88, 95% CI 0.84–0.92) and quantification (intraclass correlation coefficient range 0.79–0.93), performing similarly to interobserver variability.
Lipid plaque identification showed moderate agreement (κ=0.68, 95% CI 0.64–0.72), with an intraclass correlation coefficient of 0.79 (95% CI 0.76–0.81) for lipid arc and 0.59 (95% CI 0.55–0.63) for minimum fibrous cap thickness, exceeding interobserver reproducibility.
Model performance was limited for low-prevalence structural features, including plaque rupture.

Study Design

Type

Observational (n=100)

Blinding

Blinded expert readers (3 readers blinded to each other's interpretations and to automated analysis results)

Multicenter

No

Structured PICO

Does the OCT-AID artificial intelligence algorithm accurately perform automated full-vessel segmentation and plaque quantification compared to manual expert consensus in a real-world cohort of patients undergoing clinically indicated intracoronary OCT?

P

Population

100 consecutive patients (mean age 66 years, 24% female) undergoing clinically indicated intracoronary OCT at a single US center, providing 2,560 analyzable cross-sectional frames.

E

Exposure

Automated pixelwise full-vessel segmentation of intracoronary OCT pullbacks using a deep learning-based artificial intelligence algorithm (OCT-AID, based on nnU-Net version 2) operating on a pseudo-3-dimensional input (±3 OCT images) to distinguish lumen, guidewire artifact, intima, media, lipid plaque, calcium plaque, side branch, plaque rupture, thrombus, microvessel, and background.

C

Comparator

Reference standard obtained through manual OCT image analysis by expert readers (5 investigators with 5 to 17 years of experience; each pullback read independently by 2 readers, with discrepancies resolved by a 3rd investigator) at 2 mm intervals (every 10 frames).

O

Outcome

Frame-level agreement between automated AI-based OCT image analysis and the expert consensus reference standard for feature identification (presence vs. absence of lipid plaque, thin-cap fibroatheroma, lipid-rich plaque, calcified plaque, side branch, plaque rupture, thrombus, and microvessel) measured by unweighted Cohen's kappa, and plaque quantification (minimum fibrous cap thickness, lipid arc, minimum calcium depth, maximum calcium thickness, and calcium arc) measured by intraclass correlation coefficient (ICC).surrogate

Main Result

Effect estimate: Kappa 0.88 vs 0.90 (95% CI 0.84-0.92)

Absolute Event Rate: 0.88% vs 0.9%

p-value: p=0.022

External validation in a consecutive real-world cohort demonstrates that AI-based automated OCT segmentation achieves plaque identification and quantification performance comparable to or exceeding expert interobserver agreement, supporting its clinical feasibility.

Limitations

Single-center external validation dataset from an institution with high OCT expertise, potentially limiting generalizability.
Modest patient sample size (100 patients) and frame-wise rather than pixel-wise overlap evaluation due to the laborious nature of manual interpretation.
Exclusion of frames with substantial image artifacts, mostly at the outer edges of the pullback.
Reference standard relied on manual expert OCT interpretation rather than coregistered histopathology.
Retrospective study design without real-time prospective clinical evaluation.
Single-center study with large experience in clinical OCT, potentially limiting generalizability to other centers.
Modest number of evaluated patients (n=100) due to the laborious nature of manual OCT image interpretation.
Expert analysis was limited to frame-wise classification and quantification rather than pixelwise segmentation, precluding an evaluation of within-frame overlap.
Frames with substantial artifacts precluding the establishment of a reference standard were excluded.
Limited algorithm performance for low-prevalence features such as plaque rupture and thrombus.

Ask about this paper

Answers are grounded in this paper and cite their sources.

Implication

May support standardized OCT plaque analysis in research; leaves open prospective validation before clinical adoption.

Abstract

BACKGROUND: Artificial intelligence (AI) allows automated evaluation of intracoronary optical coherence tomography (OCT) images. However, algorithms are mostly developed and validated on well-curated data sets, which may not represent real-world data. We sought to externally validate a previously developed algorithm performing full-vessel segmentation for OCT in an unselected consecutive real-world data set. METHODS: This was a retrospective, single-center, external validation study comprising 100 consecutive patients undergoing clinically indicated OCT. A previously developed AI algorithm (OCT-AID) was used for automated pixelwise labeling of OCT images, distinguishing among lumen, guidewire artifact, intima, media, lipid plaque, calcium plaque, side branch, plaque rupture, thrombus, microvessel, and background. The AI-based predictions were compared on a frame level to the reference standard obtained through manual OCT image analysis by expert readers. RESULTS: Among 2560 analyzable frames, the agreement between the automated OCT image analysis and the reference standard was excellent for calcified plaque identification (κ=0.88 [95% CI, 0.84-0.92]) and quantification (intraclass correlation coefficient values ranged between 0.79 and 0.93), with a performance close to interobserver variability. For lipid plaque identification and quantification, the model performance was reasonable (κ=0.68 [95% CI, 0.64-0.72]; intraclass correlation coefficient for lipid arc, 0.79 [95% CI, 0.76-0.81]; intraclass correlation coefficient for minimum fibrous cap thickness, 0.59 [95% CI, 0.55-0.63]) and largely superior to interobserver variability. The algorithm performance for low-prevalence features (e.g., plaque rupture) was limited. CONCLUSIONS: AI-based fully automated evaluation of OCT images is feasible with performances consistent with interobserver variability in a real-world data set of consecutive patients, supporting generalizability of the proposed methodology.

Discussion

No takes yet. Share an insight, caveat, or question.

Cite This Study

Volleberg et al. (2026) conducted an observational in Coronary artery disease undergoing clinically indicated intracoronary optical coherence tomography (OCT) (n=100). AI-based automated OCT full-vessel segmentation algorithm (OCT-AID) vs. Manual OCT image analysis by expert readers (consensus reference standard and interobserver variability) was evaluated on Frame-level agreement for calcified plaque identification between AI algorithm and expert consensus vs interobserver agreement (Kappa 0.88 vs 0.90, 95% CI 0.84-0.92, p=0.022). OCT-AID automated OCT segmentation achieved excellent agreement with expert consensus for calcified plaque and outperformed interobserver agreement for lipid plaque.

synapsesocial.com/papers/6af7c94e33d7b13b78f560e1 https://doi.org/10.1161/jaha.125.049353

View Full Paper

Ask AI

Bookmark

Share

Real‐World External Validation of Artificial Intelligence‐Based Full‐Vessel Segmentation for Intracoronary Optical Coherence Tomography

Key Points

Study Design

Structured PICO

Main Result

Limitations

Ask about this paper

Implication

Abstract

Discussion

Cite This Study

Real‐World External Validation of Artificial Intelligence‐Based Full‐Vessel Segmentation for Intracoronary Optical Coherence Tomography

Key Points

Study Design

Structured PICO

Main Result

Limitations

Ask about this paper

Implication

Abstract

Discussion

Cite This Study

Keep reading

Keep reading