Answers are grounded in this paper and cite their sources.
Currently, leaderboards are often used to evaluate natural language processing (NLP) systems and in particular large language models.In this paper we argue why we should step away from leaderboards and follow a more inclusive approach both in developing as well as in evaluating models.The focus of evaluation should be on the complete context in which the system operates.To accomplish this, researchers should take an inclusive approach and take note of developments in multiple scientific fields (from NLP to communication science).
No takes yet. Share an insight, caveat, or question.
Braggaar et al. (2024) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: