Measuring impacts: a scoping review of healthcare impact evaluations

Background With the increasing use of the term “impact evaluation” in healthcare and the absence of an exhaustive review on this general theme, this research aims to map available evidence and methods associated with impact evaluations in healthcare by conducting a scoping review. Methods This exhaustive review included peer-reviewed studies of healthcare interventions with no restrictions on language or time of publication. Results In total, 324 studies met the inclusion criteria from 4372 single registries retrieved from Medline, Embase, Scopus, WoS and Econlit in August 2024, with no time restriction. Only ex-post studies were identified; as expected from guidelines, most studies used counterfactuals (58%) and only 7% did not use any comparison. Furthermore, natural experiments or quasi-experiments were the most applied designs (37%), followed by observational (26%) and experimental (17%) designs. Conclusions Impact evaluations of healthcare interventions seem to be predominantly associated with methods of strong comparison (counterfactuals) designs as seen in guidelines; however, there are exceptions.