Answers are grounded in this paper and cite their sources.
Software engineering has built an infrastructure for preserving research artifacts, and a set of automated checks that certify them. Those checks read metadata: that a link resolves, that a persistent identifier exists, that a record carries a license field. The artifact itself is a container, and the container is almost never opened. A check that reads the wrapper rather than the deliverable cannot fail in the direction that matters: it reports clean whether or not the archive holds anything usable. We open the containers. Every zip archive carries its own index at its end, inside the deliverable rather than in the record describing it, and an HTTP Range request retrieves that index alone. Using this technique, long established outside this setting, we recover the complete file listing of an archived replication package without downloading it. Across 329 archived artifacts we transferred 0.184% of the archive bytes, 886.1 MB against 471.0 GB, and recovered every file name. The host's own API does publish a file listing, but it lists the files of the record, a single entry for 78% of these: a median of 1 against a median of 138 paths inside the archive. The listings show what the metadata cannot, and they split in two. The properties that help a person read an artifact are common: 83% contain a README and 78% contain at least one runnable file. The properties that determine whether it can be reused are close to a coin flip: 53% contain a license and 47% a dependency manifest of any kind, the one component a reader cannot reconstruct unaided. Presence is a floor, not a certification, and these figures are upper bounds on compliance rather than estimates of reusability. A strict-versus-generous rule ablation keeps README highest and the manifest lowest throughout, breaks the two-tier grouping under the strictest rules, and moves the manifest rate between roughly a quarter and a half of archives, so we report that figure as an order of magnitude. Pairing archives against the repositories they name required verifying provenance first: 20 of 119 records citing a repository are not snapshots of it, and comparing against a merely cited dependency manufactures divergence. We release the auditor, the frozen sampling frame, the per-archive listings and the controls, including a full-download cross-check confirming that the cheap method and the expensive one agree. We argue that content-level checks belong in artifact evaluation, because the checks in place today are structurally unable to notice an empty box.
No takes yet. Share an insight, caveat, or question.
Simarjot Khanna (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: