Excess significance and power miscalculations in neurofeedback research

We use the term "repeated measures (mixed) ANOVA" to describe a study design with one independent measure and one repeated measure (e.g., two independent groups measured at two time points).GPower uses the term "ANOVA: Repeated measures" for this type of design.4 Kohl and coauthors (including me-RTT) were unaware of this default option in GPower.I was only led to become aware of this issue when reading the fMRI-nf review and noticing a surprising amount of statistical power. 5Kieslich (2020) provides a detailed explanation of how GPower calculates f for the different options for repeated measures (mixed) ANOVAs.The GPower team informed us that Cohen proposed Cohen's f for between-subjects designs and that this is how GPower defines f when using the default option, regardless of the ANOVA design the user selects. 6We performed our calculations in R.They are all available in our open code.We reproduce Table 3 from Tursic et al. and Table 5 from Kohl et al. using the sample sizes and statistical tests which each review provides in their supplementary material (i.e., we did not re-extract this information from the original studies or check if the statistical tests in the original studies were appropriate).The WebPower package we used assumes a correlation of 0.7 for repeated measures for ANOVAs, which is slightly more conservative than the 0.8 correlation used in the reviews.