ABSTRACT Differentially private deep learning has become essential for training models on sensitive data. However, existing methods like DP‐SGD suffer from inefficient privacy budget utilization and layer heterogeneity. Uniform clipping and noise across all layers overlook differences between entry layers and deep layers in gradient scale and noise tolerance, which skews signal to noise ratios and leaves fragile layers excessively perturbed. We propose differentially private hierarchical noise‐aware learning (DP‐HNAL) to address this challenge. DP‐HNAL combines selective update mechanisms with hierarchical noise allocation. Our method allocates per layer clipping thresholds proportionally to parameter counts. We introduce a noise‐aware loss function that enables the model to learn layer specific noise coefficients during training. This loss couples actual injected noise variance with layer parameter norms. It creates a feedback loop where weak layers automatically receive less noise and strong layers carry more. We further introduce an adaptive global target that adjusts noise intensity based on validation acceptance rates. This mechanism dynamically balances privacy protection with learning capacity. We evaluate DP‐HNAL on MNIST, Fashion‐MNIST, CIFAR‐10, and IMDB under identical privacy budgets. Experiment results demonstrate that DP‐HNAL consistently outperforms existing methods, achieving improvements of maximum 5% on complex datasets.
Hu et al. (Wed,) studied this question.