Funding grant proposals for scientific research: retrospective analysis of scores by members of grant review panel

OBJECTIVE: To quantify randomness and cost when choosing health and medical research projects for funding. DESIGN: Retrospective analysis. SETTING: Grant review panels of the National Health and Medical Research Council of Australia. PARTICIPANTS: Panel members' scores for grant proposals submitted in 2009. MAIN OUTCOME MEASURES: The proportion of grant proposals that were always, sometimes, and never funded after accounting for random variability arising from differences in panel members' scores, and the cost effectiveness of different size assessment panels. RESULTS: 59% of 620 funded grants were sometimes not funded when random variability was taken into account. Only 9% (n = 255) of grant proposals were always funded, 61% (n = 1662) never funded, and 29% (n=788) sometimes funded. The extra cost per grant effectively funded from the most effective system was A18, 541 (£11, 848; €13, 482; 19, 343). CONCLUSIONS: Allocating funding for scientific research in health and medicine is costly and somewhat random. There are many useful research questions to be addressed that could improve current processes.