Answers are grounded in this paper and cite their sources.
This study examined whether the validity and reliability of scores differed between online and face-to-face administrations of instruments that varied in length and measured typical response and maximum performance. A nonrandomized comparative desing was used. Typical-response data (5-item Perseverance Scale; 30-item Perceived Academic Self-Efficacy Scale) were collected from 568 middle school students (335 paper-and-pencil, 233 online); achievement data (9-item reading and 25-item listening tests) were collected from 94 university preparatory students (50 paper-and-pencil, 44 online). Model fit and measurement invariance were examined using confirmatory factor analysis and multiple-group confirmatory factor analysis for ordinal data respectively; differential item functioning (DIF) using Mantel-Haenszel and logistic regression, requiring statistical significance and a non-negligible effect size, and Cronbach’s alpha coefficients using Feldt’s F-test. Both typical response scales showed acceptable fit in both modes. The Perseverance Scale reached metric invariance while the Self-Efficacy Scale reached only configural invariance. Alpha did not differ for the Perseverance Scale (0.78 in both modes) or the long achievement test (0.80 online vs. 0.67 face-to-face, p = 0.097), but was significantly lower online for self-efficacy (0.83 vs. 0.88, p = 0.011) and lower face-to-face for the short achievement test (0.79 vs. 0.59, p = 0.027). No item met the DIF criterion in the short form; in the long form, none showed uniform DIF, but 3 of 25 items (12%) showed non-uniform DIF. Psychometric equivalence across modes could not be assumed and should be evaluated empirically for each instrument and population.
No takes yet. Share an insight, caveat, or question.
ÖZYETER et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: