Key points are not available for this paper at this time.
The Research Assessment Exercises (RAE) conducted in the UK have attractedvarious types of published response. These include citation analyses and a review of the public reception of the RAE 1996, which included a brief critique of the citation studies. This paper develops the critique. Largely unexplored issues in the theory or assumptions of bibliometrics, e.g. the level of citation which corresponds to a quantum of research quality, are found to emerge in the studies. A weak, and unsatisfactorily treated, correlation between citation aggregates and research quality for individual entities is revealed. The proposal to replace informed peer review by citation analysis is regarded as highly unrealistic. Productive uses for citation analysis in research evaluation are suggested. A historically rare instance of correlation between rankings derived from citation aggregates and from real world peer review has been revealed by the studies. The future value of citation analysis could be to inform, but not to determine, judgements of research quality. A combination of methods is advocated for future studies of the RAEs. Information science must attend to considerations of value, as well as using established techniques, if it is to avoid marginalisation.
Julian Warner (Fri,) studied this question.