The power of randomization tests to detect individual intervention effects. Insights from a Monte Carlo study

Despite an overall agreement on the usefulness of single-case experimental designs (SCED), there is no consensus on the most suitable approach to analyzing the resulting single-case data. Previous research indicates that reliable and rigorous analysis methods in SCED are crucial for the sensitivity, specificity, and robustness of subsequent inferences on the effectiveness of interventions. Among others, randomization tests depict a promising approach to data analysis, which received less attention in research and practice than more popular analytical approaches. So far, no clear picture of the exact conditions under which randomization tests yield high statistical power and low alpha-error rates has emerged. In the present paper, we applied several Monte Carlo studies to scrutinize the influence of various design conditions on the sensitivity and specificity of a randomization test for detecting meaningful within-person changes. We focused on the number of measurement times of each phase, the size of the intervention effect, the reliability of the measurements, the trend effect, and the number of cases in multiple-case designs. The results of our study demonstrate that a) the power of randomization tests for analyzing single-case data is high and the alpha error rate is low when cases have at least ten measurements in Phase A and the intervention effect is large, b) randomization tests are robust against data trends, and c) multiple-case designs rapidly increase the power, allowing even for minimal designs with 15 or fewer measurements. (DIPF/Orig.)