Log analysis demonstrates detection of silent AI model transitions in local coding sessions, indicating that user metadata can verify unannounced software changes.
Users of AI coding assistants have no version string, no changelog of behavior, and no ability to roll back to a prior model. When the tool changes, the operator has only an impression, and an impression is unfalsifiable. This note describes a method for turning that impression into a measurement. Claude Code writes the model name, client version, and reasoning effort into local session logs that most users never open. A script walks the archive and reports, per session, the model served and the density of the assistant's own self-corrections. It is demonstrated on a single operator's archive of 64 sessions covering 2026-07-20 to 2026-08-26. The operator stated on 2026-07-31, from behavior alone and with no tooling, that the assistant had changed. The logs place the model boundary at 2026-07-30. The claim preceded the measurement by four weeks and was one day off. The method is the contribution. The worked example demonstrates that it functions, and it is reported with its limitations intact: the sample is unbalanced (5 sessions against 52), the workload differs across the boundary, an independent control signal does not confirm it, and the pattern and the test are drawn from the same data. It is a change-point observation with a date attached. It is not a quality finding, and quoting it as one is a misuse of it. Section 1 discloses the conflict of interest in full: the paper was written by the system it measures. Reproduction: node scripts/model-drift.mjs ~/.claude/projects/ . The script reads local files only and writes nothing. Any user of the same tool already has the archive. The measurement worth taking is the reader's own.
No takes yet. Share an insight, caveat, or question.
Nathan Hall (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: