shedding-bench: Retention and Recovery Under Context Compaction in Coding-Agent Sessions

A coding agent that works long enough loses its context window. Context management then determines what the session can retain. We measured what remained answerable after context shedding, which recovery consultations the harness detected, and the run-level resources consumed. Our first instrument planted a ledger of announced values in the work and asked for them at the end. It produced a clean result, but the result was an artifact: the run that scored had answered from its own compaction summary, one entry above the exam. The benchmark rewarded the mechanism whose failure it was built to measure. This paper describes the replacement instrument. It measures retention and recovery of session provenance under context shedding: which files arrived together, in which pass, and next to which others. The harness generates that grouping and the session experiences it as part of its work. Nothing announces it, and it is not re-derivable from the checkout. Collection uses a stale project note dropped into the checkout with wrong values, which the agent updates with its own tools. Grading joins the agent's output to content hashes of what the harness delivered; no language model decides a primary number. Recovery remains reachable and observed rather than forbidden: the session's own transcript is off the read tool and outside the working directory, consulting it is detected from shell-command text and counted by phase, and every run's score is reported beside the channel detected and the resources the run consumed. The paper reports one formative campaign, eight graded runs of one plan over the curl repository under native compaction and a lossless-folding subject, and states what a formative campaign cannot establish: no ranking, no construct validation, and no answer-level attribution of recovery. The record holds the paper, its LaTeX and figure sources with the d3 figure pipeline, the claims contract that bounds every factual statement, the evidence builders and their outputs for the ledger-era postmortem and the formative campaign, the imported campaign seals with their manifest and hashes, and SHA-256 manifests. The instrument itself is open source at github.com/shaneconner/shedding-bench under the MIT license; the paper and author-created artifacts are CC BY 4.0, and the pinned curl source retains the curl license. This is a preprint and has not been peer reviewed.