Statistical Power for Moderation in Three-Level Multisite Individual Randomized Trials and Consequences of Ignoring a Level of Nesting

Three-level multisite individual randomized trials (MIRTs), in which individuals are nested within teachers and schools and randomly assigned to treatment or control conditions, provide a robust framework for assessing overall intervention effects and moderation at multiple levels. This study develops a statistical framework for designing three-level MIRTs to evaluate moderated treatment effects and examines the impact of ignoring one level of nesting on Type I error rates and statistical power. We illustrate the framework with an example of an online tutoring program and derive formulas for statistical power and the minimum detectable effect size difference. These formulas are validated through Monte Carlo simulations, which also demonstrate the risks of ignoring nesting. Finally, we introduce a software tool to facilitate power analysis for moderation in three-level MIRTs and summarize key findings.