Implications of Image-Level Duplication in Public Glaucoma Datasets

Importance Medical artificial intelligence (AI) can be trusted only if validated in patients not used for development. In glaucoma AI, a few public datasets from 1 laboratory serve as those independent cohorts. Objective To determine whether 3 public glaucoma datasets used for external validation and fairness evaluation are independent patient cohorts. Design, Setting, and Participants This cross-sectional dataset audit and record-linkage analysis was conducted in July 2026 using the publicly distributed files of Harvard-GDP (1000 eyes), Harvard-GF (3300 eyes), and Harvard-FairVision (10 000 eyes), all from a single laboratory. Records were linked by exact visual field and demographic fingerprints, with chance modeled from the within-dataset collision rate and permutation. All 1302 candidate pairs underwent direct image comparison. Main Outcomes and Measures Number of shared eyes and image-level identity of optical coherence tomography (OCT) scans distributed under different identifiers. Results Harvard-GF and Harvard-FairVision matched on 1302 eyes against 1.9 expected by chance. Direct comparison confirmed 1031 (79.2%; 95% CI, 76.9%-81.4%) as the same 3-dimensional OCT volume (cosine similarity ≥0.9999; 88% of voxels exactly equal), at least 31% of the 3300 Harvard-GF eyes, a lower bound. Duplication was similar across the 3 Harvard-FairVision splits (78.5%-80.8%). Harvard-GDP and Harvard-GF shared 22 eyes matching exactly on all 52 total-deviation points and mean deviation, all in the Harvard-GF validation split; Harvard-GDP distributes no OCT volume, so its overlap rests on weaker evidence. No file carried a patient identifier, and no release disclosed the overlap; among the 64 of 105 citing studies whose full text we obtained, none reported it. Across 5 training runs, a classifier trained on 1 dataset and evaluated on the other gave an area under the curve of 0.90 (95% CI, 0.83-0.94) where 78.5% (95% CI, 74.0%-82.5%) of test eyes were training data, against 0.78 (95% CI, 0.72-0.83) out of sample (inflation, 0.12; 95% CI, 0.06-0.18). Inflation occurred in every racial subgroup and was larger where out-of-sample performance was lower (Spearman ρ = −0.57; permutation P = .02). Conclusions and Relevance These findings suggest that the examined datasets were not independent cohorts. Treating them as independent benchmarks inflates reported performance and might mask fairness gaps through data leakage and nonindependent validation.