Comparing Quality in Academia Utilizing Data Envelopment Analysis: The Relationship Between Quantitative and Qualitative Indicators at Institutions of Higher Education

Israel’s system of higher education, like many national research evaluation systems worldwide, contends with a constant tension between quantitative metrics and the need to promote significant scientific excellence. This article explores the relationship between quantitative and qualitative indicators at academic institutions by utilizing the data envelopment analysis (DEA) technique. Based on analysis of 52 academic institutions (15 in Israel and 37 in other countries), we examine the performance of 47 Israel Prize laureates in the exact sciences (1978–2023), according to the criteria of Israel’s Planning and Budgeting Committee (PBC) budgeting model. The main finding reveals that only 19% of Israel Prize recipients rank in the upper quintile under the current model. Adjusting the indicators by period and discipline raises this rate to 67%, indicating a widening discrepancy between quantitative indicators and the system’s ability to detect outstanding scientific excellence. This disconnect is particularly pronounced in theoretical fields such as physics and mathematics. We propose a combined model, based on DEA principles, that balances quantitative and qualitative indicators. Additionally, we introduce a quality indicator based on Ioannidis’ c-score index, which ranks the top 100,000 scientists worldwide, with adjustments for citations, self-citations, co-authorship, and research fields. We propose a quality indicator (QI) measuring the average contribution of a researcher’s publications to their c-score ranking. Validation against Nobel Prize laureates, MIT researchers, and Israel’s established universities shows results compatible with institutional reputations. While based on Israeli data, these findings have implications for research evaluation systems worldwide that employ similar quantitative metrics for funding allocation, particularly regarding the balance between efficiency measurement and support for breakthrough science.