Randomized trial evaluates token-budgeted knowledge graph effectiveness in AI coding agents, suggesting improved retrieval methods.
AI coding agents spend most of their context re-reading source files to orient themselves — a cost paid on every prompt that grows linearly with repository size. Code knowledge graphs are an increasingly popular remedy, but recent systems bundle many graph signals at once, leaving open which signals actually improve retrieval enough to justify their tokens. We present cgraphy, an open-source, language-agnostic knowledge-graph layer served over the Model Context Protocol (MCP), and use it as an instrument to answer that question. cgraphy contributes (1) importance-weighted, token-budgeted subgraph extraction; (2) git co-change edges fused with the static graph under leakage control; (3) an agent-self-enrichment protocol in which the host agent writes persistent, content-hash-keyed semantic summaries — no server-side LLM or API keys; (4) edit-loop tools (budgeted symbol reading, blast-radius impact analysis, diff-to-symbol review context) with usage-aware retrieval; (5) optional hybrid retrieval fusing FTS with tiny static embeddings by reciprocal-rank fusion; and (6) two reproducible, grading-free benchmarks: a commit-derived localization suite (450 tasks, 9 repositories, 7 languages) and an issue-driven suite built from SWE-bench Lite instances evaluated at their pre-fix snapshots. Across 9 repositories, the signal stack lifts mean hit@10 from 0.602 (lexical baseline) to 0.733, with semantic fusion the largest single contributor at zero added payload; on real GitHub issues the full stack gains +15.8 points hit@5 (0.456 → 0.614 hit@10). We then close the loop with 400+ controlled runs of real agents: merely exposing graph tools costs accuracy, imperative steering costs 14 points, and only an enriched graph with substitution-style steering restores parity — yet the same deployed configuration resolves 14 vs. 8 of 57 real issues end-to-end under the official SWE-bench Docker harness (+75% relative), locating the graph's agentic value in the editing phase rather than first-file localization. cgraphy indexes kubernetes (26K files, 219K nodes) in 49s with 1.6s freshness checks; on real private codebases, orientation via the graph is 134–420× cheaper than reading the code. Code, benchmarks, predictions, and data: https://github.com/pmgarg/cgraphy
No takes yet. Share an insight, caveat, or question.
Prateek Mohan Garg (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: