Validation of a multi-video view-informed vision-language model for valvular heart disease assessment in japanese patients

Background Artificial intelligence (AI) is increasingly applied to automate echocardiographic interpretation in structural heart disease. However, external validation across diverse populations is essential prior to clinical implementation. EchoPrime, a multi-view vision–language AI model for echocardiographic analysis, was predominantly trained in Western cohorts, with only 7.5% Asian representation. Purpose To evaluate the agreement between AI-derived and human-reported grading of valvular heart disease (VHD) severity in a Japanese cohort and to investigate mechanisms underlying clinically relevant discordance. Methods We retrospectively analysed consecutive patients undergoing transthoracic echocardiography between 2011 and 2020 for mitral regurgitation (MR), aortic regurgitation (AR), aortic stenosis (AS), and tricuspid regurgitation (TR). Severity was graded on a 5-point scale (0 = none/trace, 1 = mild, 2 = mild-to-moderate, 3 = moderate, and 4 = severe). AI-derived grades were compared with contemporaneous clinical reports. Agreement was assessed using quadratic weighted κ and proportion of agreement within ±1 grade. Grading transitions were visualised using Sankey plots. Cases with major discordance (Δgrade ≥3) were independently adjudicated by two expert cardiologists to identify potential causes. Results Among 517 consecutive patients, the AI model generated reports in 465 cases (89.9%). Quadratic weighted κ demonstrated low-to-moderate agreement:MR: κ = 0.118 (95% CI 0.025–0.210; p=0.011)AR: κ = 0.088 (95% CI 0.001–0.177; p=0.049)AS: κ = 0.288 (95% CI 0.162–0.408; p<0.001)TR: κ = 0.157 (95% CI 0.073–0.235; p<0.001) Agreement within ±1 grade was higher: 70.5% (MR), 81.9% (AR), 86.9% (AS), and 76.3% (TR) [Figure 1A]. Figure 1B shows an example for MR, where most discrepancies occurred between adjacent grades. Major discordance (Δgrade ≥3) was observed in 163 patients (35.1%), where AI overestimation in 92 cases (19.8%) and underestimation in 101 cases (21.7%) (30 had both). Error patterns were valve specific. In MR, AR, and TR, discordance was frequently associated with colour Doppler misinterpretation and valve misidentification. In AS, major discrepancies were consistently related to failure to incorporate continuous-wave (CW) Doppler waveform analysis (Figure 2). Conclusion Although agreement within ±1 grade was generally acceptable, clinically significant discordance occurred in approximately one-third of cases. Valve misidentification, eccentric regurgitant jets, and incomplete Doppler integration appear to be key limitations. Targeted algorithm refinement and incorporation of comprehensive Doppler analysis could help broader clinical generalisation.Figure 1Figure 2