Assessing Self-Perceived Critical Thinking in AI-Mediated Academic Tasks: Development and Initial Psychometric Evidence for the Critical Thinking with AI Scale for Students (CTAI-S)

Artificial intelligence (AI) is commonly used in university academic work, yet existing instruments mainly assess AI literacy, readiness, or levels of permitted use rather than the critical-thinking processes students activate when working with AI. This study developed the Critical Thinking with AI Scale for Students (CTAI-S) and provided initial psychometric evidence for its use. An initial 15-item questionnaire was administered to 271 university students, and one item was removed after item analysis. Parallel analysis favored one predominant factor (KMO = 0.927), and a theory-guided three-factor representation was also examined. Ordinal confirmatory factor analyses (WLSMV; n = 257) yielded scaled CFI = 0.964 and RMSEA = 0.072 for the one-factor model, and CFI = 0.977 and RMSEA = 0.059 for the equivalent correlated-three-factor and second-order models; robust indices were less favorable, and latent correlations were very high (0.85–0.88). Total-score reliability was high (ω = 0.898), whereas the third grouping showed low reliability (ω = 0.557). Mean responses were above the scale midpoint (M = 3.97), with Reflective Judgment and Self-Regulation being highest (M = 4.07). The CTAI-S is best interpreted as a global self-report measure of critical thinking in AI-mediated academic tasks; independent validation is needed before dimensional or diagnostic applications.