Quantitative-based lung ultrasound aeration scoring improves reliability compared with coalescence-based scoring: a prospective validation of the 2025 ESICM–ESPNIC criteria

The 2025 ESICM–ESPNIC consensus endorsed a quantitative-based criterion (≤ 50% vs. > 50% pleural-surface involvement) over the previous coalescence-based criterion for distinguishing lung ultrasound (LUS) score 1 from score 2. Whether this change improves scoring reliability across operators with different experience levels has not been prospectively evaluated. We compared the two criteria using a simultaneous multi-rater design. In this prospective single-center study, consecutive eligible adult ICU patients underwent standardized 12-region LUS within 24 h of admission. All clips were independently scored under both criteria by three blinded raters spanning the operator spectrum: an attending intensivist (> 8 years’ LUS experience), a respiratory therapist (> 3 years), and a resident with no prior LUS experience. The primary outcome was inter-rater reliability, quantified by ICC(2,1), quadratic-weighted Cohen’s kappa, and Gwet’s AC1 (prespecified primary overall agreement measure). Secondary outcomes were intra-rater reliability, score-distribution patterns, and clinical construct validity against PaO₂/FiO₂ and ICU length of stay. The final cohort included 180 patients, yielding 2,160 regional observations. The quantitative-based criterion yielded higher inter-rater reliability than the coalescence-based criterion (three-rater Gwet’s AC1 0.947 vs. 0.922; p < 0.001), with the largest gains among less experienced rater pairs. The direction of the findings was unchanged after excluding regional clips scored 0 by all three raters under both criteria. Intra-rater ICCs were higher with the quantitative-based criterion for the respiratory therapist and resident, but similar for the attending intensivist. Under the coalescence-based criterion, scores 1 and 2 were used less often; score 2 accounted for only 1.4–4.4% of regions, and 59–72% of quantitative-based score 2 regions were assigned score 3. Both criteria correlated similarly with PaO₂/FiO₂ (Spearman ρ = −0.39 to − 0.44). The quantitative-based criterion improves inter-rater reliability of LUS scoring across experience levels and increases intra-rater ICCs for less experienced raters, without compromising clinical construct validity. These findings provide prospective multi-rater evidence supporting Statement 4.2 of the 2025 ESICM–ESPNIC consensus.