Examining the value added by committee discussion in the review of applications for research awards

We examined a process for evaluating research fellowship proposals in which each was assigned to two members of a review committee for in-depth assessment. Before the committee meeting the reviewers scored the proposal against weighted criteria using benchmarked scales and a detailed rating guide. At the meeting they presented their reviews and then received questions and comments from colleagues. Subsequently each committee member assigned a score reflecting their overall appreciation of the proposal. We observed committees at work, analysed pre-meeting and post-discussion scores and considered feedback from reviewers. Our results suggest that committee discussion and rating of proposals offered no improvement to fairness and effectiveness over and above that attainable from the pre-meeting evaluations. Copyright , Beech Tree Publishing.