Applied Psychometrics: The 3-Faced Construct Validation Method, a Routine for Evaluating a Factor Structure

The “3-faced construct validation method” is a routine for establishing the validity and reliability of an existing scale when adapted in a different cultural context from the context initially developed. This routine can also be used for the initial validation of a newly developed scale. This is essentially a construct validation procedure based on a sample-splitting. The sample is randomly split into three parts, 20% for EFA, 40% for an exploratory CFA and 40% for a cross-validating CFA. The cases per variable threshold is set above 5:1, preferably above 10:1 (minimum conditions) and the first approximately 20% subsample emerges (adequate conditions) to evaluate EFA and Bifactor EFA models. Then the cases per variable threshold is set above 10:1, preferably above 20:1 (minimum conditions) and the 40% subsample emerges (adequate conditions) to examine alternative CFA models (ICM-CFA, Bifactor CFA, ESEM and Bifactor ESEM models). The optimal model(s) is cross-validated by a second CFA in yet another 40% subsample (equal-power CFA sample) as a protection against overfitting (over-optimization) to safeguard model replicability. Measurement invariance follows and is essentially another cross-check of the optimal model over the entire sample because the optimal model is used as the baseline model.