Answers are grounded in this paper and cite their sources.
Test of two signals for sentences that LLM coders of teacher talk moves probably get wrong: disagreement among Claude Sonnet 5.5, GPT-6 Luna and Gemini 4 Argon, and the stated confidence of Claude Sonnet 5.5. A frozen annotation harness codes 97 new passages from 8 teachers of TalkMoves Subset 2. Nothing is optimized. The primary measure is Cohen's kappa on the sentences not flagged. Confirmatory hypotheses test each signal against no flagging and compare the two directly, with cluster bootstrap over teachers and transcripts and Holm correction. Registered before the passages were drawn.
No takes yet. Share an insight, caveat, or question.
Simon Filler (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: