Answers are grounded in this paper and cite their sources.
Data lineage tools learn about data movement from integrations, so code without one, such as anad-hoc script or a notebook, moves data unseen. We study how much of this gap better capture canclose and how much lies beyond the reach of any observer of the wire. We built DCP (DataConnectivity Protocol), which captures lineage inside the PostgreSQL and Kafka client libraries ofunmodified Python programs. On programs with no lineage code, the OpenLineage client emitted 0events while DCP matched every answer key. At scale, DCP's edge-level events reached aprovenance precision of 0.895; the same lineage expressed in OpenLineage's core run model, whichrecords only a run's inputs and outputs, reached 0.480, at equal recall. On a stress set, DCP's recallfell to 0.880, and every miss had one of two causes: a table that does not record who wrote its rows,or a choice made inside the program. We classify the gap into four classes and derive a ceiling forany interceptor that only observes. We then propose the Provenance-Native Protocol, a framingprotocol over TCP in which a data-moving frame without provenance is invalid, reads return thewrites behind their rows, and writes must cite their inputs. A ledger gateway lets it run in front ofstores that keep a per-row writer id. A TLA+ model checks its guarantees and reproduces DCP'sfailure modes. We project the protocol's benefits from these measurements; its costs remain to bemeasured.
No takes yet. Share an insight, caveat, or question.
Dev Bhirin Desai (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: