An AI-based workflow achieved a higher success rate for left heart volumetric assessment than expert manual analysis, with significant improvements in LVEF and diastolic volume measurements (p=0.03).
Observational (n=46)
Does a fully automated AI pipeline improve the reliability and accuracy of left heart volumetric assessment compared to expert manual analysis in patients undergoing echocardiography?
A fully automated AI pipeline for echocardiographic left heart volumetric assessment provides superior reliability and comparable accuracy to expert manual analysis, reducing operator dependency.
p-value: p=0.03
Abstract Introduction Artificial intelligence (AI) is a powerful tool for fully automating echocardiographic analysis, that has demonstrated high accuracy and reduced variability 1–3. However, its reliability across diverse clinical scenarios remains underexplored. In routine practice, operator-dependent variability often limits study completeness and interpretability, affecting the number of analysable studies. Purpose This study aims to evaluate the reliability and accuracy of a fully automated AI-based workflow for left heart volumetric assessment, benchmarked against expert manual analysis in a real-world clinical setting. Methods A total of 925 transthoracic sequences were retrospectively collected from 46 patients (median age 60 y, IQR: 53-70; 67% male). Left ventricular and atrial volumes were calculated using a purely manual workflow and a fully automated AI pipeline, based on a combination of CNN and nnUNet architectures 4, developed to perform view classification, cardiac structure delineation, and measurements. Biplane measurements were prioritized over single plane ones. Each study was manually analysed by 2 independent sonographers (from a pool of 6; mean expertise 8.5±3.5 y), who annotated the sequences that they deemed suitable for volume quantification. Studies discarded by both sonographers were re-evaluated by a clinical committee. Reliability was quantified as the proportion of valid studies yielding clinically usable measurements. Success rates between manual and automated analyses were compared using the two-proportion z-test. Relative errors were calculated to assess accuracy and interobserver variability. Results Out of 46 total studies, 43 were deemed valid for analysis. Sonographers successfully analysed 41 studies, but only 23 of those were annotated twice, suggesting high interobserver disagreement. The AI approach successfully quantified all parameters in 38 studies and demonstrated higher success rate across all clinical metrics, with significant improvements in left ventricular ejection fraction (LVEF) and diastolic volume (p=0.03) (Table 1). Automated measurements achieved comparable accuracy to human operators (Figure 1). LVEF mean relative error of the system was 13.8±12.2% vs 13.5±13.7% interobserver variability. The clinical committee confirmed that AI interpretations were adequate in those cases discarded by both sonographers, highlighting its robustness in low-confidence scenarios. Moreover, the AI approach exhibited lower variance across all the metrics, indicating greater consistency. Conclusions This study demonstrates that AI-based echocardiography analysis can deliver superior reliability in real-world data while preserving a comparable accuracy to human experts. By successfully processing more studies, the AI system reduces operator dependency and enhances diagnostic yield. These findings underscore the potential of AI in clinical workflows to elevate diagnostic accuracy and patient care.Table 1 Figure 1
Garcia-Elcano et al. (Thu,) conducted a observational in Echocardiography analysis (n=46). Fully automated AI-based workflow vs. Expert manual analysis was evaluated on Proportion of valid studies yielding clinically usable measurements (success rate) (p=0.03). An AI-based workflow achieved a higher success rate for left heart volumetric assessment than expert manual analysis, with significant improvements in LVEF and diastolic volume measurements (p=0.03).