Probing Privacy Risks in Deep Neural Networks: A Jacobian Rank-Based Analysis of Intermediate Layers

Deep learning models are known to be vulnerable to privacy attacks that can reconstruct or infer sensitive information from the training data. While many privacy assessment methods exist, they are often computationally prohibitive or overlook the considerable information leakage occurring from the model’s intermediate layers. This paper introduces an efficient, attack-independent framework for quantifying privacy risks by analyzing the sensitivity of these internal representations. We discover that the rank of the Jacobian matrix of an intermediate layer’s output with respect to the model’s input serves as a robust proxy for privacy vulnerability. Specifically, we analyze the dynamics of the Jacobian rank throughout training, proposing 2 metrics, rank recovery (RR) and normalized rank recovery (NRR), to specifically quantify a layer’s dynamic transition from generalization to memorization. Through extensive experiments on a diverse range of computer vision models and datasets, including modern residual networks, we demonstrate a strong and consistent positive correlation between our proposed rank-based metrics and the success rates of membership inference attacks. Our findings reveal that deeper layers, which exhibit considerably higher RR, are the primary sources of privacy leakage. This work provides an actionable methodology for developers to identify high-risk layers in real time, facilitating the development of more secure and privacy-preserving machine learning systems without the need for expensive adversarial simulations.