A PRISMA Review of Large Ensemble Adjudicated Performance Literature

We conducted a systematic review of a century of scholarship on large ensemble adjudicated performance (LEAP) literature in the United States. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses method, we identified 106 articles that met our inclusion criteria. Our analysis revealed that the most common type of publication was quantitative research (51.8%), followed by nonempirical (32.1%), historical (10.4%), and qualitative articles (5.7%). The most frequent topics of LEAP scholarship were factors influencing ensemble ratings (n = 23) and assessment topics such as reliability and score distributions (n = 14). Notably, researchers’ interests (factors in adjudication and the reliability of scores) were somewhat disconnected from perennial practitioner debate over the value of competition in adjudicated performances.