The reviewer is always right: peer review of research in Medical Education

When I first heard Glenn Regehr, a member of Medical Education's international editorial board, say 'The reviewer is always right', I imagine my reaction must have been similar to those of most authors upon reading the quote: powerful and viscerally negative reflections on the times I had received feedback from (typically anonymous) reviewers who had reviewed my work, yet had seemed unqualified, unhelpful or simply obtuse. Actually, I imagine this reaction to have been less strong than those elicited from many as I tend to be an adamant supporter of peer review in all scholarly endeavours.1 Despite a favourable inclination towards peer review, it is easy to recall instances of passing through the denial and anger phases of grief upon receiving feedback on my work. Of course, since taking on the role of editor-in-chief, it is now also easy to recall instances when I have had to explain the basis for decisions made to authors who have not quite passed through the anger phase themselves after their receipt of less-than-flattering reviews. How can it be that the reviewer is always right when it is so easy to recall instances in which the reviewer has been dead wrong? Before I answer this, let me try to make absolutely clear that this editorial is not about defensively or absolutely cutting off discussions about the value of specific submissions to the journal. Mistakes can certainly be made during the process of evaluating papers and, although Medical Education is fortunate to have a tremendously talented and dedicated team of staff, deputy editors and peer reviewers, I am quite fallible. If you think a mistake has been made despite the view expressed in the title of this editorial, feel free to let us know, as everyone affiliated with the journal is much more interested in making the right decisions than in being right. More immediately, however, this editorial is written, firstly from the perspective of an author regarding how one might productively approach the construct of peer review and, secondly, in order to highlight some of the strategies we are implementing in the journal to try and strengthen the way in which peer review is utilised. So, how can it be that the reviewer is always right when it is so easy to recall instances in which the reviewer has been wrong? The answer, somewhat circularly, is that, even in those infuriating instances, the reviewer was not wrong. Carrying the conversation forward beyond his initial statement, Regehr clarified that: 'If the reviewer has misunderstood something, even if I can point to a paragraph in the text that explicitly and literally counters the reviewer's claim, then I as the author have failed to make the point clearly.' Reviewers can miss or misinterpret things, but we as authors need to remember that peer reviewers tend to read our papers much more carefully and deliberately than the general readership of any journal does. If the peer reviewer has missed something, then the odds are good that someone else will as well. As a result, the onus is on us as authors to find a way to pitch the message in a way that minimises confusion and, in turn, maximises impact (see Roediger's 12 tips for authors2 for further suggestions on how to do so). The reviewers, after all, are also our audience. For each submission the team of editors strive to find reviewers with methodological and content expertise relevant to the focus of the paper, but the database of potential reviewers as a whole is constituted of people who work in the medical education and medical education research communities. They are the people we want to read and use our research efforts and if we do not pitch our work to them in a way that makes it clear and meaningful, then we can be sure that its impact will be less than optimal, even if it is published. Thought of in this way, Regehr's rule that the reviewer is always right isn't as much a statement of fact as it is a suggestion of orientation. We all intensely (and sometimes lovingly) craft our papers until we think we have them just right. That effort makes it hard to hear or accept criticism, but it is absolutely vital to set our visceral reactions aside and try to understand how our work could be improved based on the feedback received. It used to be said that you should read reviews of your work, allow your emotions to take flight, and then place the reviews in a desk drawer and leave them there for a week. In the electronic age we may not literally use our desk drawers anymore, but still the sentiment is a useful one: allow time to pass and emotions to subside, and then return to your paper with a spirit of openly questioning how it can be made better based on the comments received. For all the flaws inherent in peer review, I have never seen a paper, whether working as an author, reviewer or editor, that was not improved as a result of going through the process. I believe so strongly in its value that I would encourage all authors to engage the peer review process prior to submitting work to any journal. If you have colleagues who are not authors on the article read and substantively critique your work prior to submission, then you stand a greater chance of correcting the flaws before the stakes are raised during journal-imposed review. Having said all that, we at the journal still believe that we have an obligation to ensure that the reviewers are always right, not at the level of controlling content, but at the level of doing whatever we can to run the process in an efficient, professional and constructive manner. In terms of efficiency and professionalism we are doing quite well: thanks to the hard work of the editorial staff, the deputy editors and those of you in the field who provide your invaluable insights as peer reviewers,3 the average turnaround time from submission to first decision continues to fall despite the process being effectively the same as it was 10 years ago.4 In terms of the credibility or constructiveness of the reviews provided, I have to admit (probably to no-one's surprise) that there is heterogeneity in the field. Most of the reviews we receive are of high calibre and include some truly exceptional critiques that warrant publication in their own right; other reviews make it clear that there are some people working in the field who have never been taught how to perform a peer review. For individuals in the latter group, we are starting the new year by offering two new supports, both of which are built in part on the advice of Goldbeck-Wood, who argued in the British Medical Journal a decade ago that 'the best reviewers concentrate on offering useful advice to authors rather than giving summary judgements to editors'.5 To facilitate the delivery of useful feedback we have created a much more descriptive review form that should inform or remind reviewers which aspects of the manuscript should be considered when generating a review. The ultimate goal, in all instances, should be to ask yourself, as the reviewer, 'Did I learn anything?' and 'How could the authors have helped me learn more?' In other words, 'What could the authors have done that would convince me of the argument they are trying to convey?' If you cannot answer that question as a reviewer, it is necessary to determine whether the problem is unsolvable, the research method is untenable, or the reviewer is irascible and unreasonable. Given the variety of methodologies and epistemologies in our field, it is impossible to come up with a comprehensive form that covers all possible contingencies, but we have put together a generic form that should prompt constructive feedback, even if comments are not required or applicable in every box for every paper. Each box is accompanied by a hyperlink that will provide novice reviewers with some indication of the specific questions they should ask themselves when trying to provide feedback to authors (see Roediger's 12 tips for reviewers6 for further advice in this regard). As we would also advise authors to consider the review form prior to submitting a manuscript, we have made the full series of questions available as a 'Guidelines for Reviewers' document that can be downloaded from http://www.mededuc.com. In educators' parlance, this is the marking scheme that will be used by reviewers and editors to assess the quality and publishability of research submissions. As a result, research mentors might also be wise to discuss the document with their protégés. The second support we would like to offer the community consists of more deliberate feedback and mentoring on how to write reviews well. If you are not on our list of reviewers and would like to be added, please write to med@mededuc.com. Whether you are currently in the database or not, if you would like explicit feedback on your performance as a reviewer and suggestions as to how you might improve it, feel free to write to the same address to make that request. We will try to match you with a paper and an editor or another experienced reviewer who can provide some mentoring in this important domain. Many of you have formal mentors who already provide this service (or would if you asked), but through this offer we hope to help those of you who do not have ready access to those types of individuals. At first blush these attempts to improve the peer review process and uptake of the advice of peer reviewers might seem self-serving and it is true that Medical Education as a journal (and, in fact, all journals in the field) stands to benefit from an increase in the number of people who provide useful reviews. Our goal in prioritising peer review is much broader, however, in that improvement in peer review, I think, represents an important step in the continued maturation of the field. Better reviews mean better papers, which implies a stronger evidential and theoretical basis on which to build our practice improvements. At the same time, you don't need to feel altruistic to want to actively engage in the peer review process. When I was starting my academic career I was advised to agree to provide a peer review as often as I was asked. The argument put to me by my mentor, Lee Brooks, was that reviewing is a great way to (a) get a broader view of the issues of interest in the community, (b) identify gaps in the way the community is thinking about particular issues worthy of further study, and (c) become established in the community in your own right by showing opinion leaders in the field that you are capable of writing timely and thoughtful peer reviews. The advice served me well and I hope that more elaborate discussion about peer review and efforts towards broadly improving the process will serve our research community equally effectively.