Understanding Private Learning From Feature Perspective
Meng Ding, Mingxi Lei, Shaopeng Fu, Shaowei Wang, Di Wang, Jinhui Xu
TL;DR
This work addresses how DP-SGD behaves when learning features in the presence of label-dependent signals and label-independent data noise. By modeling data with a multi-patch structure and analyzing a two-layer CNN under NoisyGD, the authors reveal that private training requires a higher signal-to-noise ratio than non-private training to effectively learn features, and that memorization of data-noise can persist under differential privacy, harming generalization even when training loss is small. They introduce a weight decomposition into signal and noise coefficients, establish two regimes (signal learning vs data-noise memorization) governed by $\mathrm{SNR}$ and the privacy budget $\varepsilon$, and derive training/test loss characterizations including a DP-dependent test error term $\exp((n\varepsilon)^{-1-1/q})$. Experimental results on synthetic data and CIFAR-10 corroborate the theory, underscoring the need for feature-enhancement strategies to improve $SNR$ in privacy-preserving learning. The work provides a principled foundation for understanding private learning dynamics and informs design choices for DP-trained neural networks in practical, privacy-constrained settings.
Abstract
Differentially private Stochastic Gradient Descent (DP-SGD) has become integral to privacy-preserving machine learning, ensuring robust privacy guarantees in sensitive domains. Despite notable empirical advances leveraging features from non-private, pre-trained models to enhance DP-SGD training, a theoretical understanding of feature dynamics in private learning remains underexplored. This paper presents the first theoretical framework to analyze private training through a feature learning perspective. Building on the multi-patch data structure from prior work, our analysis distinguishes between label-dependent feature signals and label-independent noise, a critical aspect overlooked by existing analyses in the DP community. Employing a two-layer CNN with polynomial ReLU activation, we theoretically characterize both feature signal learning and data noise memorization in private training via noisy gradient descent. Our findings reveal that (1) Effective private signal learning requires a higher signal-to-noise ratio (SNR) compared to non-private training, and (2) When data noise memorization occurs in non-private learning, it will also occur in private learning, leading to poor generalization despite small training loss. Our findings highlight the challenges of private learning and prove the benefit of feature enhancement to improve SNR. Experiments on synthetic and real-world datasets also validate our theoretical findings.