FedPCOS-XAI: a reliability-audited federated learning framework for polycystic ovary syndrome classification from ultrasound imaging

Introduction Deep learning studies on public ultrasound archives for polycystic ovary syndrome (PCOS) report near-perfect accuracy, but few studies check where the data came from or how the models were evaluated, and federated learning (FL) benchmarks carry the same problems. This study builds a reliability-audited federated benchmark on a widely used public archive. Methods SHA-256 and perceptual hashing of 11,784 files removed 7,788 exact duplicates; near-duplicate grouping, which also matched rotated and flipped copies, then excluded 132 images with conflicting labels, leaving 3,864 images in 2,345 groups. A 773-image test set sharing no duplicate group with the training data was held out, and the remaining images were split across five simulated clients. Eight CNNs were trained with federated averaging (FedAvg) over three seeds and selected on validation data only. Controls included centralised and local-only training and an augmentation ablation; models were compared with a paired group bootstrap, and gradient-weighted class activation mapping (Grad-CAM) faithfulness, leakage, and shortcut reliance were audited. Results Balanced accuracy ranged from 0.983 to 0.995, and no model differed from the validation-selected reference after correction for multiple comparisons; MobileNetV2 and ShuffleNetV2 sent up to 106 times less data per round than VGG-16. Federated training was within 0.006 of centralised training and 0.031–0.064 above local-only training. FP16 overflow in aggregated batch normalisation statistics and the divergence of networks without batch normalisation were reproduced and corrected. Grad-CAM was more faithful than a spatially matched random control for six of eight models. Image dimensions alone classified the test set perfectly, and five of eight models kept a balanced accuracy of 0.91–0.98 when the central half of each image was masked. Conclusions Accuracy on this archive is high, but the models can classify the images from acquisition cues outside the ovary, so the results do not show recognition of polycystic ovarian morphology. PCOS classifiers need clinically confirmed data from several sites.