Protein overlap inflates apparent generalization in aptamer-protein interaction benchmarks

Aptamers are nucleic-acid ligands that bind protein targets, enabling applications in diagnostics, therapeutics, and biosensing. Their costly experimental discovery motivates computational prediction of aptamer-protein interactions, but protein overlap between training and test data can inflate performance estimates. We audited the widely reused Li et al. benchmark and found that 579 of 580 test pairs (99.83%) share a protein with the training set. At fixed epoch 25, AptaTrans performance remained near chance under protein-family-disjoint evaluation (AUC 0.51; MCC 0.04), compared with AUC 0.87 and MCC 0.59 under pair-random splitting using the same publication-era codebase, architecture, and fixed classification threshold. We report this retrained baseline because we could not reproduce the published metrics; our audit identified test-label-based threshold selection and a distributed checkpoint that scores perfectly on the repository’s test split. Across seven conventional classifiers, mean MCC fell from 0.458 under pair-random cross-validation to 0.001 when protein-homology clusters were held out. These results show that random pair splits are insufficient evidence of generalization to unseen proteins. We release leakage-aware splits and advocate greater investment in per-target approaches: biophysically grounded pipelines when reliable target structures are available and single-experiment machine learning otherwise. This strategy generalizes the design protocol rather than the model, applying a reproducible workflow to each target. Convincing evidence of cross-target generalization will require broader public data and evaluation that excludes protein and protein-family overlap.