Answers are grounded in this paper and cite their sources.
A molecular property or reaction-yield model is useful to a chemist only when it is right on the query in front of them, yet a model’s uncertainty is rarely checked on chemistry it has not seen. We tested an accept-or-abstain protocol in which a gate, scoring each query by its similarity to the training set and by ensemble spread, returns a prediction only when it accepts the query. The gate’s threshold is set by Learn-then-Test to a target precision on a calibration fold held out by chemistry rather than at random. On public data (23,401 logP values from SangsterLogP with two overlapping external sets, and the Buchwald–Hartwig and Suzuki–Miyaura high-throughput yield datasets), we evaluated eight held-out tests against four hypotheses defined in advance. Ranking queries by the gate’s score lowered the error rate at 30% coverage by 0.16 to 0.35 on seven of the eight tests. At a target precision of 0.8, the calibration fold mattered most for yields: gates calibrated on a random fold accepted almost every reaction with an unseen additive at a precision of 0.59 to 0.75, whereas gates calibrated on held-out additives reached a pooled precision of 0.85 to 0.87 on three of the four tests, at lower coverage. At a target of 0.9, random-fold gates came within 0.05 of the target on three of the four tests. For the random forest, fusing similarity with spread did no better than spread alone, and conformal intervals calibrated on a random fold lost coverage under shift (down to 0.64). On a fluorinated-compound hold-out, the fused score ranked errors no better than chance at 30% coverage, and the held-out-calibrated gate missed its target. All four hypotheses therefore failed their success criteria. In an exploratory analysis, conformal intervals calibrated on the held-out fold recovered coverage on seven of the eight tests, at the cost of wider intervals. We release the protocol, the code and the complete results.
No takes yet. Share an insight, caveat, or question.
Oleksandr Gurbych (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: