From Executable to Demonstrable: Synthesizing API Usage Showcases for Research Repositories

A script that runs a research repository is not the same as a script that shows how to use it. Test generation targets structural coverage, and work on executability targets whether a repository can be made to run. Neither yields what a new user of a research library needs, which is a worked example. We define demonstrativeness by three explicit predicates: in-process contact with a public non-driver symbol, use of a constructed argument, and output that depends on the returned value. We record these static predicates separately from environment construction and trial execution, preventing dependency-blocked repositories from being labelled runtime successes or failures. Reaching a library’s interface requires a target model that entry-point selection cannot express; we derive symbol-level targets by Horn clauses over signatures, exports, docstrings, and documentation references. Generated scripts are read back after every successful execution, and a vacuous script can be repaired without discarding the best executable fallback. In a paired, single-run evaluation on 90 repositories, of which 24 had buildable environments, the full configuration produced 11 passing showcases against 2 for an entry-point-only configuration and made static API contact in 15 buildable environments against 7, while both retained Tier-1 success on the same 21; exact McNemar tests on the per-repository records give p=0.012 and p=0.008. Corpus-wide, the entry-point-only configuration attempted generation on 24 repositories and the full configuration on 89, with API contact in 7 and 70; that contrast is largely by design, since only the full configuration has a generation path in a dependency-blocked environment. Because the full configuration bundles API targets, read-back, and vacuity repair, these figures establish a system-level contrast, not the isolated causal effect of any component.