Answers are grounded in this paper and cite their sources.
Chain of Draft (CoD) asks a language model to reason in very short notes instead of full sentences. With few-shot examples it has retained much of the accuracy of Chain of Thought (CoT) at a fraction of the output tokens on reasoning benchmarks, although the original study also reported weaker results in zero-shot settings and on small models. We compared one zero-shot CoD instruction with one zero-shot CoT instruction when a small local model solves tasks in a multi-step tool loop. The model, qwen2.5:7b (Q4_K_M, served by Ollama, temperature 0), answered questions about a synthetic SQLite shop database with access to three tools: read-only SQL, a reference lookup and a calculator. Each task had an exact numerical answer and an intended solution involving a SQL retrieval and at least one lookup; scoring assessed only the final number. On 240 previously unused tasks drawn from the same 120 templates and database as the pilot of an earlier study, which had shown a similar gap, CoT answered 124 correctly (0.517) and CoD 48 (0.200). The difference was 0.317 (one-sided 95% lower bound 0.250; two-sided 95% CI 0.238 to 0.396), so the directional hypothesis was confirmed, and the lower bound also exceeded the prespecified smallest effect of interest of 0.15. In a descriptive comparison that was not prespecified, CoD reached a final answer on more tasks (189/240 against 171/240) yet gave far fewer correct answers. Before the main run we froze a one-test protocol in version control, without public registration; the test used a bootstrap over task templates. CoD generated about half the recorded output tokens of CoT, and 37.5% of its replies failed a prespecified proxy for the draft limit: at most five words per non-empty line before the first CALL or ANSWER directive. The result covers one model, one task family, one seed and a budget of eight replies per task, and all CoT runs preceded all CoD runs. Because the direction came from the pilot, this is a confirmation on unused tasks from the same task generator, not a discovery. Files: the note (PDF) and a reproducibility archive with the frozen protocol, task files, SQLite database, run and analysis code, raw per-task records, run logs and SHA-256 manifests; the README lists the commands for the self-tests and for recomputing all reported numbers. Use of AI tools: the protocol, analysis code and drafts of the text were written with Claude Code (Anthropic, Claude Opus 5.5) under the author's direction; OpenAI Codex, OpenAI GPT-6 Pro, Google Antigravity and Zhipu GLM-5.3 were used for automated checks. These checks are not scientific peer review. AI tools are not authors; the author takes responsibility for all content (note, section 6). Preprint, not peer reviewed. Substantive corrections will be documented in linked versions with a change log identifying the affected claims and results. Text and results: CC BY 4.0; code: MIT.
No takes yet. Share an insight, caveat, or question.
Mariusz Kulma (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: