Between-case Incidence Rate Raito for Single Case Experimental Designs with Count Outcomes: A Monte Carlo Simulation with Conditional and Marginal Models

Calculating design-comparable effect sizes in single-case experimental designs (SCEDs) is essential for research synthesis to identify evidence-based practices that integrate findings from both SCEDs and group-based designs. The field has made significant progress in developing additive design-comparable effect sizes in recent years. This study aims to evaluate the statistical properties of a newly developed proportional estimator, the between-case incidence rate ratio (BC-IRR), for SCEDs with count outcomes. Two analytical frameworks are introduced for estimating BC-IRR: generalized linear mixed models (GLMMs) and generalized estimating equations (GEEs). A large-scale Monte Carlo simulation is conducted to assess the bias of point estimate, the standard error bias, mean squared error, and coverage rate. We also demonstrate the estimation of BC-IRR using GLMMs and GEEs with real data. Based on the simulation and demonstration results, we provide recommendations for applied researchers, discuss limitations, and outline directions for future research.