Blinded vs. unblinded peer review of manuscripts submitted to a dermatology journal: a randomized multi-rater study

BACKGROUND: Submissions to medical and scientific journals are vetted by peer review, but peer review itself has been poorly studied until recently. One concern has been that manuscript reviews in which the reviewer is unblinded (e.g. knows author identity) may be biased, with an increased likelihood that the evaluation will not be strictly on scientific merits. OBJECTIVES: The purpose of this study was to compare the outcomes of blinded and unblinded reviews of manuscripts submitted to a single dermatology journal via a randomized multi-rater study. MATERIALS AND METHODS: Forty manuscripts submitted to the journal Dermatologic Surgery were assessed by four reviewers, two of whom were randomly selected to be blinded and two unblinded regarding the identities of the manuscripts' authors. The primary outcome measure was the initial score assigned to each manuscript by each reviewer characterized on an ordinal scale of 1-3, with 1 = accept; 2 = revise (i.e. minor or major revisions) and 3 = reject. Subgroup analysis compared the primary outcome measure across manuscripts from U.S. corresponding authors and foreign corresponding authors. The secondary outcome measure was word count of the narrative portion (i.e. comments to editor and comments to authors) of the reviewer forms. RESULTS: There was no significant difference between the scores given to manuscripts by unblinded reviewers and blinded reviewers, both for manuscripts from the U.S. and for foreign submissions. There was also no difference in word count between unblinded and blinded reviews. CONCLUSIONS: It seems, at least in the case of one dermatology journal, that blinding during peer review does not appear to affect the disposition of the manuscript. To the extent that review word count is a proxy for review quality, there appears to be no quality difference associated with blinding.