Abstract Artificial intelligence has drawn on cognitive and educational psychology since its origins, and contemporary training pipelines for large language models and related systems (curated data curricula, staged prompting supports, feedback from human judges, and benchmark-driven evaluation) make this connection unusually concrete. Drawing on Messick’s (1995) unified theory of validity, we use educational psychology to derive a set of validity checks that distinguish robust competence from support dependence, proxy optimization, and rater-specific compliance. We organize the discussion around five domains where the analogy is especially revealing: curriculum design, scaffolding and instruction, social learning via human feedback, assessment and validity, and ethical alignment. For each domain, we map a familiar educational construct onto concrete training interventions such as data selection and sequencing, prompting and tool support, preference learning and other feedback loops, benchmark design, and reward modelling. For each construct, we identify what would count as evidence that an apparent training gain is, or is not, what it appears to be. We conclude with implications for model development practice, including documenting curriculum and scaffolding decisions, treating benchmarks and reward models as high-stakes assessments that can invite proxy optimization and “teaching to the test,” and making explicit how assessment targets are revised as models and deployment contexts change.
Wang et al. (Tue,) studied this question.