Answers are grounded in this paper and cite their sources.
Public brain tumour MRI collections are routinely reported with near-perfect classification accuracy, yet they carry no patient identifiers and their train/test overlap is rarely quantified. We present a duplicate-aware, reproducible frozen-feature benchmark on BRISC 2025 (6000 contrast-enhanced T1-weighted slices; glioma, meningioma, pituitary tumour, no tumour). A perceptual-hash audit found 1352 candidate near-duplicate pairs, with 439 of them crossing the official train/test split, and 104 training images that are pixel-identical to test images; a visual review of sampled pairs showed that most candidates were other slices of the same scan, with an estimated 5–10% false matches. Removing redundant copies and every training image connected to the test set left 5477 images, a split that is duplicate-aware but not patient-independent. Under an identical linear probe, ImageNet-supervised ResNet-18/50 and self-supervised DINOv2 ViT-S/14 were competitive: DINOv2 was numerically ahead (macro-F1 77.3 ± 0.1%, macro-AUROC 93.2%), but no pairwise difference survived Holm correction. In-pool validation overstated official test macro-F1 by 9.7–13.5 points; control experiments showed that this gap is not an artefact of the filtering but mainly reflects a distribution difference between the official partitions. Meningioma recall stayed at 50–53%, and all linear heads were under-confident. We release the splits, audit tables and per-image predictions.
No takes yet. Share an insight, caveat, or question.
Vajjhala et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: