Key points are not available for this paper at this time.
Abstract Evaluation is central to science, technology, and innovation (STI) systems, shaping the allocation of recognition, resources, and career opportunities. A widely held assumption is that evaluation processes, such as awards, grants, and fellowships, approximate underlying merit, although in a somewhat biased and/or noisy manner. This Research Brief challenges that assumption. Drawing on social choice theory and insights from the political economy of science, it argues that collective evaluation of complex, multidimensional research outputs cannot reliably produce stable or coherent rankings of “quality.” Instead, outcomes emerge from structurally constrained aggregation processes, committee dynamics, and strategic considerations. Awards and similar recognitions should therefore be understood as low-resolution, path-dependent signals that reflect a combination of quality, visibility, institutional positioning, and political alignment. The Brief outlines implications for how evaluation outcomes are interpreted, how researchers respond to them, and how STI systems might be designed to mitigate distortions.
Nicolai Foss (Wed,) studied this question.