Traceprop: End-to-End Provenance-Guided Data Attribution for Auditable Machine Learning

Traceprop is an open-source Python library providing the first unified system for end-to-end data provenance in machine learning pipelines, connecting raw source files through preprocessing, through model training, to individual predictions. Existing data attribution methods [Koh and Liang, 2017, Park et al., 2023, Engstrom et al., 2024] identify which training samples influenced a prediction but operate in isolation from the data pipeline. Existing computation lineage tools (MLflow, DVC, TensorFlow MLMD) track artifact-level provenance but do not descend into the computation graph or connect to gradient-level attribution. Traceprop fills this gap by introducing a computation-level lineage layer that integrates natively with gradient-based attribution. A single Traceprop query answers: “This model made prediction X: which rows in which source files, through which preprocessing steps, most influenced that prediction, and can we reduce that influence without retraining?” We demonstrate: (1) sub-1% lineage overhead in production op-mode at 106+ array elements (1.007×on macOS, 0.979×on Linux); (2) Traceprop- LL achieving LDS 0.622 ±0.180 on tabular data (UCI Adult Income, logistic regression) at 0.22 s on CPU, and Traceprop-LL achieving LDS 0.0168 on CIFAR-2/ResNet-9 vs. TRAK’s 0.0290 at 266×lower wall-clock cost (2.6 s CPU vs. 691 s GPU); (3) provenance-guided approximate unlearning exceeding the retrain-from-scratch gold standard (forget-set loss 0.425 vs. gold 0.401, vs. 14% gap closed for random unlearning) with a test accuracy drop of only 0.5 percentage points (0.915 vs. 0.920). Traceprop directly addresses EU AI Act Article 26 audit trail obligations for high-risk AI systems, whose backstop enforcement date is 2 December 2027. The library is available at https://pypi.org/project/traceprop/