Learning on the record: the observability–authenticity paradox in AI-era STEM assessment

Generative artificial intelligence tutors produce detailed records of how learners question, reason, and revise, and these interaction traces are increasingly available to teachers and institutions. This commentary examines what happens to the evidential value of interaction traces when they are used to assess learning in science, technology, engineering, and mathematics (STEM) education. I describe an Observability–Authenticity Paradox: the features that make interaction traces informative, such as exploration, expressed uncertainty, and error, are the features that learners have most reason to suppress or manage once they expect the traces to be evaluated. In terms of argument-based validity, such reactivity to evaluation weakens the extrapolation from interaction observed under evaluative conditions to learning activity that takes place without evaluation; it can also reduce the exploration on which learning depends. The threat is greatest when traces generated during learning are used for grading or credentialing without learners knowing when and how, and when inferences rest on behaviors that learners can choose to display or withhold. I distinguish this situation from teachers’ ongoing formative monitoring, consider whether assessment could instead reward productive failure, and explain why differential reactivity is a fairness issue in the measurement sense. I conclude by proposing three design principles (declared, episodic, and learner-owned assessment) and illustrating them with STEM tasks, drawing examples mainly from science education.