Evaluating Robust Tests for Longitudinal Invariance Testing

Testing parameter constraints in CFA under non-normality is routinely done with robust chi-square difference tests, yet the finite-sample behavior of the available procedures remains unsettled. Pavlov et al. (Citation2020) identified the Mplus mean- and variance-adjusted test DMLMV as the most accurate established procedure in a two-model longitudinal CFA design. Here, we extend their study in three ways: we add penalized eigenvalue (PE) methods; we vary three estimation choices that Pavlov et al. held fixed—Satorra’s (2000) direct estimator of UD, the unbiased estimator of Γ, and the reweighted least-squares (RLS) base statistic; and we broaden the distributional conditions. While the performance of DMLMV was acceptable in the smaller model (p=10), it was too liberal in the larger model (p=30). Our results suggest that PE and the scaled-and-shifted (SS) test, combined with the 2000 method, deserve priority in longitudinal invariance testing. In our simulations, size-adjusted power was very similar across all procedures. In terms of Type I error control, we recommend RLS-based PE with the unbiased Γ estimator, and the RLS-based SS test, as the preferred procedures.