Misuse of AIC as a Substitute for Statistical Tests in Generalized Linear Models

AIC (Akaike information criterion) is widely used for data analysis in ecology. Although AIC was designed to identify the better statistical model for prediction, this criterion is frequently used to identify the true relationship as a substitute for statistical tests. When an explanatory variable does not affect the response variable () in truth in generalized linear models (GLMs), the rate at which the model with the explanatory variable produces the minimal AIC was determined. This error rate is similar to the Type I error rate in hypothesis testing. When calculated numerically, the error rate was close to 0.157, which is the asymptotic value expected from the theory of the likelihood ratio test, irrespective of the link function and error structure in GLMs. Even very large sample sizes (100,000 or 1,000,000) do not prevent inflation of the error rate. The error rate increases with the number of explanatory variables when none of the explanatory variables affect in truth. Similar results were obtained in AICc, a modified form of AIC for small sample sizes. In conclusion, AIC or AICc should not be used as substitutes for statistical tests.