Traceprop: End-to-End Provenance-Guided Data Attribution for Auditable Machine Learning

Traceprop is an open-source Python library providing the first unified system for end-to-end data provenance in machine learning pipelines, connecting raw source files through preprocessing, through model training, to individual predictions. Existing data attribution methods Koh and Liang, 2017, Park et al., 2023, Engstrom et al., 2024 identify which training samples influenced a prediction but operate in isolation from the data pipeline. Existing computation lineage tools (MLflow, DVC, TensorFlow MLMD) track artifact-level provenance but do not descend into the computation graph or connect to gradient-level attribution. Traceprop fills this gap by introducing a computation-level lineage layer that integrates natively with gradient-based attribution. A single Traceprop query answers: “This model made prediction X: which rows in which source files, through which preprocessing steps, most influenced that prediction, and can we reduce that influence without retraining?” We demonstrate: (1) sub-1% lineage overhead in production op-mode at 106+ array elements (1.007×on macOS, 0.979×on Linux); (2) Traceprop- LL achieving LDS 0.622 ±0.180 on tabular data (UCI Adult Income, logistic regression) at 0.22 s on CPU, and Traceprop-LL achieving LDS 0.0168 on CIFAR-2/ResNet-9 vs. TRAK’s 0.0290 at 266×lower wall-clock cost (2.6 s CPU vs. 691 s GPU); (3) provenance-guided approximate unlearning exceeding the retrain-from-scratch gold standard (forget-set loss 0.425 vs. gold 0.401, vs. 14% gap closed for random unlearning) with a test accuracy drop of only 0.5 percentage points (0.915 vs. 0.920). Traceprop directly addresses EU AI Act Article 26 audit trail obligations for high-risk AI systems, whose backstop enforcement date is 2 December 2027. The library is available at https://pypi.org/project/traceprop/