Answers are grounded in this paper and cite their sources.
Deep-learning classifiers for pneumonia on chest radiographs are usually chosen and reported on data from a single source, and whether the chosen model, its decision threshold and its probability estimates hold elsewhere is rarely tested. We trained 15 convolutional and transformer backbones in 28 configurations, three seeds each, on 18,678 radiographs from the Radiological Society of North America (RSNA) pneumonia dataset, and fixed the best single configuration, an ensemble of the five best configurations, decision thresholds and a calibration map on 4,003 validation radiographs. The frozen models were tested on 4,003 held-out RSNA radiographs and on two external public cohorts: 5,856 paediatric radiographs and 3,364 adult studies from Indiana University. A self-supervised DINOv2 transformer fine-tuned at a low learning rate was the best single configuration. The ensemble reached an area under the receiver operating characteristic curve (AUC) of 0.904 (95% confidence interval 0.893–0.914) internally, 0.925 (0.917–0.932) in children and 0.888 (0.867–0.909) in Indiana adults. Validation rankings of the configurations predicted their adult external ranking (Spearman ρ = 0.92) much better than their paediatric one (ρ = 0.49). The threshold fixed on validation detected 84% of pneumonia cases internally but only 59% in children and 39% in Indiana adults; re-estimating it on 10% of each external cohort restored sensitivity to 83% and 79%. Calibration fitted on validation held in Indiana but not in the high-prevalence paediatric cohort, and all models were less accurate on anteroposterior than posteroanterior films. External validation of such classifiers should report thresholds and calibration, not discrimination alone.
No takes yet. Share an insight, caveat, or question.
KUMAR et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: