Answers are grounded in this paper and cite their sources.
Automated cyclic-peptide discovery runs a design–make–test–learn loop in which a model chooses what to synthesise and measure next, so the whole enterprise rests on a benchmark that can tell real learning from an artefact. That is harder than it sounds: an apparent gain can come from the starting predictor or a nearby measured analogue rather than from feedback; an improvement in one chemical series or laboratory need not survive the move to the next; and a control meant to catch a rigged benchmark can just as easily condemn a sound one. Here we present a benchmark framework that pulls these questions apart with coordinated controls — frozen-model, one-shot, shuffled-feedback and retrieval comparisons for the source of a gain; acquisition versus held-out-source ranking for its transfer; and planted-valid and planted-shortcut tests for the validity check itself. Applied to the largest public permeability and binding datasets, the controls are revealing. Most of the selection advantage arrives before any feedback, and the reproducible 47.3-hit gain proves to sit almost entirely in a single data source. Held-out- source ranking improves only marginally (Spearman gain 0.075) and cannot be separated from shuffled feedback. The registered validity gate falsely condemns a quarter to a third of clean benchmarks, where a null-calibrated gate condemns just 3.0–8.3% and still catches planted shortcuts (area under the receiver-operating curve, AUROC, 0.65 and 0.70); the same scrutiny leaves a generation benchmark inconclusive, its physics verifier below its own floor (0.071 against 0.10). The framework turns benchmark interpretation into an empirical object of study: trust comes from controls, not from a headline metric.
No takes yet. Share an insight, caveat, or question.
Zhang et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: