Analysis reveals that only 15% of proposals were categorized as RoE, suggesting a gap in systematic research methods.
Many evaluators may be conducting and presenting research on evaluation (RoE) without realizing it. Proposals accepted to present at the AEA 2019 conference were analyzed for whether or not they were RoE (i.e., systematic, empirical, and focused on evaluation). RoE proposals were then coded using a framework by Mark (2008) to determine the common trends across those identified as RoE. Non-RoE proposals were coded for why they were not RoE, including that they were not systematic, empirical, and/or focused on evaluation. A total of 15% of the 732 proposals analyzed were coded as RoE; most proposals examined evaluation activities using the descriptive mode of inquiry. Furthermore, most non-RoE proposals were coded as not RoE because they presented an evaluation rather than a research study on evaluation or because they were reflections on evaluation. This study presents multiple opportunities and future directions for RoE.
No takes yet. Share an insight, caveat, or question.
Wanzer et al. (2025) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: