Animal experiments are often purely exploratory, with little to no data available to support the planning phase. Nonetheless, ethical guidelines demand scientifically sound planning, particularly regarding sample size determination based on biometric criteria such as power analysis or precision of effect estimation. The experimental designs are typically complex, involving numerous experimental groups and adaptive steps, which complicates statistical planning. To date, existing statistical approaches for animal experiments have largely ignored this complexity. Despite widespread recognition that effect sizes in animal studies are often biased, poorly replicable, and rarely translate well to clinical trials, little emphasis has been put on this remarkable gap between experimental research and statistical planning. We demonstrate that common design practices in animal experiments introduce substantial error in effect size estimation, even if properly adjusted for inflated type I error rates and false discovery rates. To address this, we propose a simulation-based approach to quantify the estimation error and to classify its magnitude compared to a reference design. We advocate for a two-stage experimental approach-comprising a screening and a confirmation phase-using robust mixture priors for effect size estimation. Our simulation study compares the operating characteristics of various designs and illustrates how optimal designs can be selected. Additionally, we present supporting software tools aimed at facilitating communication with nonstatistical collaborators.
Zocholl et al. (Sun,) studied this question.