Validity and reliability establishment for questionnaire-based research studies

Questionnaire-based surveys are widely used in health, social, behavioral, nursing, management, and educational sciences, yet a substantial share of studies rely heavily on Cronbach’s alpha as the sole reliability index, a practice increasingly identified as inadequate and often misinterpreted. This primer provides step-by-step guidance for assessing validity and reliability across four commonly mishandled psychometric domains. Content validity requires item-level CVI (I-CVI ≥ 0.78) and scale-level CVI (S-CVI/Ave ≥ 0.90) from ≥ 6 experts using CVR and CVI. Construct validity is assessed via Exploratory Factor Analysis (EFA; factor loadings ≥ 0.40, n ≥ 200) and Confirmatory Factor Analysis (CFA; CFI > 0.95, RMSEA < 0.08, SRMR < 0.08), alongside convergent (AVE > 0.50; CR > 0.70) and discriminant validity (HTMT < 0.85). Internal consistency benchmarks are α ≥ 0.70 acceptable, α = 0.80–0.89 good, α ≥ 0.90 excellent, and α > 0.95 indicating redundancy, with McDonald’s omega (ω) and 95% CI recommended as a superior alternative. Test-retest reliability favors ICC over Pearson’s r, with ICC < 0.50 poor, 0.50–0.74 moderate, 0.75–0.89 good, and ≥ 0.90 excellent, assessed in n = 30–100 over 1–4 weeks. The full validation follows a five-phase framework—item generation, expert review, pilot testing (n = 30–60), main validation (n = 200–500), and cross-validation aligned with COSMIN and APA Standards (2014). No single index is sufficient; multi-method assessment is essential for rigorous, publishable research.