Normative specification of SuffBench, a benchmark format for context sufficiency in code editing. Each instance records a real code change (base and gold commits, task text with its source, gold and test patches), the write-set and interface write-set, fail-to-pass and pass-to-pass (frame) test sets with flakiness exclusions, backward and forward slices from a named dependency graph, a verdict-based coverage slice, an optional mutation slice, a closure check of the dependency relation, per-context expressibility of the gold patch, token counts, cutoff flags, and provenance. The specification includes a JSON Schema, a datasheet template, and the command-line contract of a planned subset checker and context compiler for Go. It labels verification relevance, not generation sufficiency. No instances are released with this specification. Companion to the note DOI 10.5281/zenodo.23282196.
No takes yet. Share an insight, caveat, or question.
Jude Castiel Johansen (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: