Content Analysis of Statistical Power in Educational Technology Research: Sample Size Matters

In educational technology research, most studies are conducted to explore the effectiveness of using technology to improve teaching and learning. Priori power analysis enables researchers to determine sufficient sample size for achieving adequate statistical power during research planning. Observed power analysis is carried out on completed studies to estimate statistical power. While priori power analysis is recommended for sample size estimation, observed power analysis has been criticized for being incorrect and misleading. To understand current practices of power analysis in the field, we conducted a content analysis on five years’ publications in Educational Technology Research and Development from 2014 to 2018, a total of 178 articles. Our findings showed that only two articles (1.1%) reported a priori power analysis and seven articles (4.0%) reported observed power although it is not recommended. To facilitate sample size determination during research planning, we generated sample size tables for various t tests and ANOVAs from G*Power. Best practice recommendations to conduct and report educational technology research are provided.