Assessing the robustness of normality tests under varying skewness and kurtosis: a practical checklist for public health researchers

Abstract Background Many statistical methods used in public health research, namely t-tests, ANOVA correlation and regression, rely on the assumption of normality. Violation of the normality assumption can severely lead to biased parameter estimates, reduced test power, and impact the reliability and validity of the findings, impacting the real-world evidence. An attempt to provide guidelines for choice of appropriate tests for assessment in public health data analytics is being made in this article. Methods This study aims to compare the performance of 13 commonly available normality tests in various software’s, namely Shapiro–Wilk, Shapiro-Francia (Regression-Based tests), Lilliefors, Cramer Von Mises, Anderson–Darling (Empirical distribution-based test), Jarque–Bera, Adjusted Jarque Bera Test, Robust Jarque–Bera, D’Agostino & Pearson, D’Agostino Skewness, D’Agostino Kurtosis, Gel Miao Gastwirth (Moment-Based test), and Pearson Chi-Square (Chi-square-based test). These tests were evaluated based on empirical Type I error and power across varying sample sizes, skewness, and kurtosis using Monte Carlo simulations with non-normal data generated via the Fleishman method, reflecting slight to significant deviations in terms of skewness and kurtosis. Results For moderately skewed data with low kurtosis, the D’Agostino Skewness and Shapiro–Wilk tests perform better across all sample sizes while Robust and Adjusted Jarque–Bera tests are preferable at higher kurtosis. In highly skewed data, Shapiro–Wilk is most effective, with Shapiro-Francia and Anderson–Darling improving with larger samples. For symmetric data, RJB and GMG are robust choices, with GMG preferred at higher kurtosis. Findings from two real-world datasets also support the simulation results. Conclusion Performance of Normality tests are significantly influenced by sample size, skewness, and kurtosis. The findings of this study contribute to improving statistical practices in public health research by providing a practical, evidence-based checklist for selecting appropriate normality tests based on these key sample characteristics.