How would you recognise a good review?

Academic journals would be impossible and sometimes unsafe without the contributions of volunteer reviewers. But reviews come in all shapes and sizes, and are not always a reliable aid to decision. This editorial distils my experience as author, reviewer and editor about the characteristics of a useful review. It addresses the task and responsibilities of the reviewer and the characteristics of the review. The review of a research article requested by an editor has three broad purposes: to advise the editor, to encourage the author and to protect the reader. Many reviewers assume that their role is to decide whether an article should be published. This is a serious misunderstanding since the reviewer cannot access or assess the range of relevant information available to the editor, such as the number of accepted articles awaiting publication, the range of topics and how long they have been waiting. Such misunderstandings can cause difficulties when reviewers make recommendations that turn out to be at odds with an editorial decision. Advising the editor is a valuable task, but decisions to publish remain with the editorial team. Good reviewers encourage authors by recognising the strengths of the submission and highlighting possible improvements. Making a contribution to research quality improvement is a long-standing expectation of members of the research community and often requires time and effort. Timely reviews are always welcome. Reviewers protect the reader by assessing the novelty of the submission, its technical soundness, and its conciseness or verbosity. Readers may include clinical trainees, experienced clinicians, managers, policymakers or researchers. Poor or ill-informed reviews may result in the rejection of useful material and the publication of unreliable findings. Reviewers must be sure that they are competent to review the submission. This may imply knowledge of particular research literature or familiarity with contemporary research techniques. Doubts about either of these should be discussed with the editor before accepting the request. Good reviews have the following characteristics. They are comprehensive and address each element of the paper, including the title, abstract, key questions, introduction, methods, results and discussion. Editors must be confident that the whole manuscript has been read carefully. Some readers may only see the title and abstract, and this needs to be an accurate summary of the paper. Good reviews make clear what is new and perhaps important in a submission and how it might contribute to knowledge and practice. Sometimes this will require some focused literature searching by the reviewer. Failure to cite recent papers in the bibliography may indicate that writers have not consulted the literature adequately. Journals that repeat established knowledge or established misconceptions are unlikely to fare well. Good reviews state whether data are handled appropriately and statistics are well chosen, and whether evidence supports findings and recommendations are reasonable. Some papers still fail to distinguish statistical association from causation or assume that perceptions are more than answers to an interviewer's questions. Qualitative papers without discussion of the analysis performed are not acceptable, nor are review papers without adequate discussion of methods. Awareness of standards for statistical reporting or accepted publication guidelines is a good starting point for assessing the quality of a paper. Many of these standards are available on the website, www.equator-network.org (accessed 10 June 2014). Good reviews distinguish critical core findings from unnecessary detail, frequently removing clutter from papers and simplifying where appropriate. Good reviews make clear recommendations for improvement. This should not focus on spelling and grammar but rather on whether the standard of English is suitable for publication. It is quite acceptable to comment that the paper requires editing before review, and in many cases editors will identify such papers before requesting reviews. Good reviews are empathetic. They recognise that writing for publication is a challenge, and in some cases an extreme challenge for those writing in a second or third language. They also recognise that junior clinicians and early career researchers may be under pressure to publish, and need help to develop research and writing skills. Providing reviews should also be instructive for the reviewer. While the contents of a manuscript sent for review are confidential, the reviewer should see the review process as an opportunity to learn something new, whether about the substance of the paper, the methods employed, the context of the study or the policy environment in which the research took place. Perhaps we need to recalibrate our thinking. Anonymous reviewers are the unsung heroes of the academic community, and as such they deserve our thanks. Without their assistance, it would be even more difficult to distinguish the good from the bad and the important from the commonplace. In this edition, we thank our reviewers for 2013 by listing them on page 145. Thank you all for your generosity.