Answers are grounded in this paper and cite their sources.
Modern deep learning models have driven significant performance breakthroughs in document key information extraction (KIE) systems. At the same time, however, their data-driven nature exposes these systems to a new class of privacy vulnerabilities. Since KIE systems directly interact with sensitive information in documents, they are potential targets for privacy attacks. Despite this, research on privacy-preserving multimodal document KIE remains scarce. To address this gap, we conduct the first systematic investigation into privacy-preserving training for multimodal document KIE models, comparing differential privacy (DP), federated learning (FL), and differentially private federated learning (DP-FL) approaches. Through extensive experimentation across six benchmark datasets, we demonstrate that while DP training typically causes substantial performance degradation, this can be effectively addressed through: (1) the use of pretrained document foundation models; (2) DP-suited hyperparameter tuning; and (3) task-specific pretraining that aligns target entity types with the private data distribution. In particular, this combination allows up to 77–93% retention of non-private baseline performance in both standalone and federated setups (via global server-side optimization) while maintaining strong privacy guarantees ( ε ∈ ε ∈ {8, 20}). Notably, task-specific pretraining alone contributes major performance gains of 2–6.5% when using out-of-distribution public data (partial entity-type overlap) and up to 20% when using in-distribution synthetic data (full overlap). To empirically assess the privacy strength of each approach, we evaluate them against membership inference attacks, and show that non-private KIE models can be highly vulnerable, with the attack reaching near-perfect success rates on some datasets, whereas both FedAVG and DP substantially mitigate this vulnerability. Finally, through systematic analysis of hyperparameters, architectural choices, and pretraining strategies, we provide practical guidelines for achieving optimal privacy-utility trade-offs, enabling practitioners to deploy privacy-preserving document KIE systems in real-world applications.
No takes yet. Share an insight, caveat, or question.
Saifullah et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: