Answers are grounded in this paper and cite their sources.
As a conference version of the author's report to Nature and Science about their published AI papers since 2015, this report charges three major frauds: (1) Post-Selections Using Test Sets (PSUTS), (2) Post-Selections Using Validation Sets, and (3) Hiding bad-looking data.This experimental protocol exaggerates the prediction accuracy.This report suggests an appropriate protocol, explains reasons for the protocol, and why what the papers have done is inappropriate and therefore yields grossly exaggerated prediction accuracies.The fraud charges below are applicable to whole systems and system components, and in all learning modes, including supervised, reinforcement, evolutionary, swarm, reservoir, and adversarial learning modes, since the concept of the experimental protocol is all the same-about fitting sets, validation sets, and test sets.In particular, this allegation of fraud is independent of the detailed applications and algorithms, including the Turing Prize 2018, the Nobel Prize in Physics 2024, and the Nobel Prize in Chemistry 2024.As a report to the Editor-in-chief, the author intends to make this report concise and free from mathematics, but the mathematical basis is available in [73].Due to (a) space limits, (b) the large number of papers charged, and (c) many papers do not mention the Post-Selection step, only detailed evidence of the alleged frauds of a few papers is presented in the paper.Facing a flood of Post-Selection misconduct, this paper further presents a theory about the exaggeration rate of Post-Selection misconduct.
No takes yet. Share an insight, caveat, or question.
Juyang Weng (2025) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: