Ten reasons why journals must review manuscripts before results are known

Ware & Munafò 1 provide a timely overview of questionable research practices (QRPs) in the life sciences and how they can be prevented. Evidence of QRPs is unequivocal: we now know that many fields are corrupted by low reproducibility and a high rate of false discoveries. The aetiology of this problem is equally clear: the practices that are best for science as a whole—rigour, replication and transparency—are out of step with the careerism that drives academic scientists to publish neat and novel results punctuated by pithy interpretations 2. This tabloid culture not only populates the literature with avoidable error, it sabotages the ability of science to self-correct 3. Solving this problem will require us to realign the incentives that motivate individual scientists with the wider needs of scientific discovery. The first, and arguably most critical, step in accomplishing this goal is to recognize that the problem exists. Thanks to the pioneering efforts of many scientists 4-10 that battle is largely won, although its gravity remains undiscovered or overlooked in several corners of science. The second step is to implement top–down reforms at the level of journals and funders. The third and ongoing stage is to monitor those reforms as they progress, adjusting them as needed to optimize gains and minimize any negative consequences. One cornerstone in the edifice of reform is the Registered Reports initiative, which is now offered by many journals 11-15 but not yet Addiction. As Ware & Munafò point out, Registered Reports represent a significant departure from traditional peer review by assessing study protocols prior to data collection and analysis. Registered Reports enshrines the philosophy that hypothesis-driven science should be judged by the importance of the research question and the robustness of the method, and never based on the outcome of hypothesis testing. The Registered Reports initiative offers at least 10 benefits for science. (1) It prevents publication bias because the results are not known during initial peer review. (2) Registering hypotheses in advance stops the pernicious but common practice of authors presenting a hypothesis derived from the data as though it was a priori 9. (3) It prevents significance chasing (also known as P-hacking) by requiring authors to pre-specify, in detail, their protocols for terminating data collection and analysing data. (4) It boosts the reproducibility of all findings, whether positive or negative, by setting high standards for prospective statistical power. (5) It increases transparency by requiring data deposition in a public archive following study completion. (6) It safeguards the publication of serendipitous findings by ensuring that editors cannot reject papers based on the outcome of hypothesis tests. (7) It incentivizes replications by provisionally accepting them before authors invest the (often substantial) resources required to repeat previous experiments. (8) It enables editors and specialist reviewers to help improve proposed methods before they are implemented. (9) It explicitly welcomes exploratory (unregistered) data analysis while also clearly distinguishing the outcomes of exploration from pre-registered hypothesis testing. Finally, (10) it fulfils an important pedagogical role in reinforcing the value of precise a priori predictions and valid statistical practices. Registered Reports is not intended as the only solution for improving reproducibility, but the model is readily adaptable to a range of scenarios. For instance, one emerging challenge is whether analysis of existing data could be pre-registered. A potential solution to this problem is to consider existing data sets as split-half ‘discovery’ and ‘replication’ samples. Under such a system, authors would submit a proposed analysis of retrospective data. If the proposal passes pre-study review and is provisionally accepted, the journal would then decide (using a random algorithm) which half of the data is the discovery sample and which is the replication sample. Given the effectively infinite number of ways in which any large data set can be bifurcated, this system would prevent authors pre-registering an analysis in which the outcome was already known. Since Registered Reports was launched in 2013 it has been adopted by 11 journals and counting. However, many journals, including Addiction, are yet to offer the format. Concerned editors cite risks to the journal's impact factor, perceived threats to exploratory research or the technical challenges in coordinating peer review. All such concerns have been addressed at adopting journals, and the Center for Open Science hosts a detailed FAQ covering these issues, and more, for both journal editors (https://osf.io/8mpji/wiki/faq:foreditors) and authors (https://osf.io/8mpji/wiki/faqs). Ware & Munafò have drawn attention to the problem of QRPs—a problem that journals have been complicit in generating and reinforcing. The duty now falls on journals, and journal editors, to help provide solutions. Registered Reports is one such solution and there is no reason for any credible journal in a field suffering from QRPs not to offer the format at the earliest opportunity. The author is section editor of Registered Reports at Cortex and AIMS Neuroscience, and chair of the Registered Reports Committee supported by the Center for Open Science (https://osf.io/8mpji/wiki/home/). At the time of publication these positions are unremunerated.