Abstract Evaluating the impact of interventions in small, high-risk areas is challenging due to sparse data and the difficulty of estimating counterfactual outcomes. This study compares statistical approaches that leverage information sharing, including Bayesian small area estimation (SAE) and disease mapping (DM) models, to assess their performance in impact evaluation. Using placebo studies —a type of simulation based on real data with modified treatment assignments—, we evaluate data-adaptive, mixed-effect, autoregressive, spatiotemporal, and fixed-effect models under realistic conditions, highlighting their strengths and limitations. These insights are further contextualized through a case study of the Garrett Lee Smith (GLS) Suicide Prevention Program in a sample of rural counties where data sparsity poses significant challenges. Results demonstrate that models incorporating information sharing improve predictive accuracy and uncertainty quantification when structured patterns exist in the data. However, in settings with extreme sparsity, the benefits of information sharing are limited. By integrating simulations and real-world applications, this study underscores the importance of model choice in impact evaluations and explores strategies for addressing data sparsity in small areas.
Garraza et al. (Mon,) studied this question.