A study that says an AI tutor improved learning needs to say one more thing: was the AI still there when learning was measured? If it was, the score describes the tool and the student together. This audit asks a narrower question than "does AI help": can a reader tell, from the paper, which situation applied? The corpus is the union of the primary studies behind three recent meta-analyses of AI-assisted learning, 107 studies. I retrieved 83 full texts by a fixed cutoff and coded one outcome measure per study against a frozen coding manual. Three language models, called by fixed identifiers, coded the 63 non-pilot papers with a performance outcome; every quotation they returned was string-checked against the paper, and all 378 passed. Two human coders coded the ten pilot papers and agreed 10 of 10 on the primary variable. One human coded a seeded random sample of 15 papers without opening the model answers. Across the 73 performance-outcome papers, 51 to 56 (69.9% to 76.7%) do not state whether the AI was available during the outcome assessment; the range covers five papers the models split on. Over all 107 studies, counting the 24 not retrieved, the range is 51 to 80 (47.7% to 74.8%). Of the 16 papers that do say, 11 had the tool available and 5 did not. I propose a six-item reporting checklist and one design rule: every learning-effect study includes at least one outcome measured with the AI absent, and says so. The audit was not preregistered, the extraction was model-assisted, and I sell a measurement product in this area. Each limit is set out in the paper, and every number can be recomputed from the per-call records.
No takes yet. Share an insight, caveat, or question.
Kumar Sumbhav Srivastava (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: