A Simulation Study on the Interaction Effects of Underfactoring and Nuisance Parameters on Model Fit Indices

Latent measurement models pose the challenge of specifying the correct number of latent factors. Researchers may underestimate data complexity, neglecting factors in confirmatory factor analysis, typically evaluated using indices like the CFI, RMSEA, and SRMR. Little is known about these indices' behavior in the case of underfactoring misspecifications and their interaction with nuisance parameters. Therefore, our extensive simulation study examined the sensitivity of the CFI, RMSEA, and SRMR to underfactoring based on up to 428,433 datasets with sample sizes from 150 to 1,350 covering 72 different configurations of multi-factorial model conditions. To summarize, results showed that underfactoring affected all three fit indices, with the effect diminishing as the number of latent variables increased. Additionally, a decreasing mean loading size reduced the impact of underfactoring on RMSEA and SRMR. Consequently, common cut-offs, such as those by Hu and Bentler (1999), often led to false positive results, particularly with RMSEA and SRMR.