Attack-Aware Noise Calibration for Differential Privacy

TL;DR

It is empirically demonstrate that calibrating noise to attack sensitivity/specificity, rather than $\varepsilon$, when training privacy-preserving ML models substantially improves model accuracy for the same risk level.

Abstract

Differential privacy (DP) is a widely used approach for mitigating privacy risks when training machine learning models on sensitive data. DP mechanisms add noise during training to limit the risk of information leakage. The scale of the added noise is critical, as it determines the trade-off between privacy and utility. The standard practice is to select the noise scale to satisfy a given privacy budget $\varepsilon$. This privacy budget is in turn interpreted in terms of operational attack risks, such as accuracy, sensitivity, and specificity of inference attacks aimed to recover information about the training data records. We show that first calibrating the noise scale to a privacy budget $\varepsilon$, and then translating ε to attack risk leads to overly conservative risk assessments and unnecessarily low utility. Instead, we propose methods to directly calibrate the noise scale to a desired attack risk level, bypassing the step of choosing $\varepsilon$. For a given notion of attack risk, our approach significantly decreases noise scale, leading to increased utility at the same level of privacy. We empirically demonstrate that calibrating noise to attack sensitivity/specificity, rather than $\varepsilon$, when training privacy-preserving ML models substantially improves model accuracy for the same risk level. Our work provides a principled and practical way to improve the utility of privacy-preserving ML without compromising on privacy. The code is available at https://github.com/Felipe-Gomez/riskcal

Figures (6)

• Figure 1: Test accuracy (x-axis) of a privately finetuned GPT-2 on SST-2 text sentiment classification dataset (top) and a convolutional neural network on CIFAR-10 image classification dataset (bottom). The DP noise is calibrated to guarantee at most a certain level of privacy attack sensitivity (y-axis) at three possible attack false-positive rates $\alpha \in \{0.01, 0.05, 0.1\}$. See \ref{['sec:experiments']} for details.
• Figure 2: Benefits and pitfalls of advantage calibration.
• Figure 3: Calibration to attack TPR (i.e., $1 -$FNR) significantly reduces the noise scale in low FPR regimes. Unlike calibration for attack advantage, this approach does not come with a deterioration of privacy for low FPR, as it directly targets this regime.
• Figure 4: Trade-off curves obtained via our method in \ref{['alg:get-beta']} provide a significantly tighter analysis of the attack risks, compared to the standard method of interpreting the privacy risk for a given $(\varepsilon, \delta)$ with fixed $\delta < 1/n$ via \ref{['eq:dp-to-f']}. The trade-off curves are shown for three runs of DP-SGD with different noise multipliers in the language modeling experiment with GPT-2. The dotted line - - shows the trade-off curve which corresponds to perfect privacy.
• Figure 5: The increase in attack sensitivity due to calibration for advantage is less drastic for Gaussian mechanism than for a generic $(\varepsilon, \delta)$-DP mechanism.

Theorems & Definitions (29)

• Definition 2.1
• Definition 2.2
• Definition 2.3
• Definition 2.4
• Proposition 2.1: dong2019gaussian
• Proposition 2.2: kairouz2015composition
• Definition 3.1
• Proposition 3.1
• Definition 3.2
• Theorem 3.3: Accounting for advantage and $f$-DP with PLRVs