Significance Tests and Estimates for R2 for Multiple Regression in Multiply Imputed Datasets: A Cautionary Note on Earlier Findings, and Alternative Solutions

Whenever multiple regression is applied to a multiply imputed data set, several methods for combining significance tests for and the change in across imputed data sets may be used: the combination rules by Rubin, the Fisher z-test for by Harel, and F-tests for the change in by Chaurasia and Harel. For pooling itself, Harel proposed a method based on a Fisher z transformation. In the current article, it is argued that the pooled based on the Fisher z transformation, the Fisher z-test for , and the F-test for the change in have some theoretical flaws. An argument is made for using Rubin’s method for pooling significance tests for instead, and alternative procedures for pooling are proposed: simple averaging and a pooled constructed from the pooled significance test by Rubin. Simulations show that the Fisher z-test and Chaurasia and Harel’s F-tests generally give inflated type-I error rates, whereas the type-I error rates of Rubin’s method are correct. Of the methods for pooling the point estimates of no method clearly performs best, but it is argued that the average of ’s across imputed data set is preferred.