Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Augment then Smooth: Reconciling Differential Privacy with Certified Robustness

Jiapeng Wu, Atiyeh Ashari Ghomi, David Glukhov, Jesse C. Cresswell, Franziska Boenisch, Nicolas Papernot

TL;DR

This work tackles the challenge of achieving both differential privacy (DP) and certified robustness (CR) in machine learning by showing standard DP training is insufficient for strong CR. It introduces DP-CERT, a simple, modular framework that weaves augmentation multiplicity, Gaussian input perturbations, regularization, and (optionally) adversarial training into the DPSGD pipeline, enabling CR guarantees without extra network components. Empirically, DP-CERT improves approximate certified accuracy and average certified radius on CIFAR10 under the same DP budget, surpassing prior approaches while maintaining competitive clean accuracy. The work also provides per-sample analyses linking certified radii to local Lipschitz constants and loss-surface smoothness, offering practical guidance for deploying privacy-preserving, robust models in real-world settings.

Abstract

Machine learning models are susceptible to a variety of attacks that can erode trust, including attacks against the privacy of training data, and adversarial examples that jeopardize model accuracy. Differential privacy and certified robustness are effective frameworks for combating these two threats respectively, as they each provide future-proof guarantees. However, we show that standard differentially private model training is insufficient for providing strong certified robustness guarantees. Indeed, combining differential privacy and certified robustness in a single system is non-trivial, leading previous works to introduce complex training schemes that lack flexibility. In this work, we present DP-CERT, a simple and effective method that achieves both privacy and robustness guarantees simultaneously by integrating randomized smoothing into standard differentially private model training. Compared to the leading prior work, DP-CERT gives up to a 2.5% increase in certified accuracy for the same differential privacy guarantee on CIFAR10. Through in-depth per-sample metric analysis, we find that larger certifiable radii correlate with smaller local Lipschitz constants, and show that DP-CERT effectively reduces Lipschitz constants compared to other differentially private training methods. The code is available at github.com/layer6ai-labs/dp-cert.

Augment then Smooth: Reconciling Differential Privacy with Certified Robustness

TL;DR

This work tackles the challenge of achieving both differential privacy (DP) and certified robustness (CR) in machine learning by showing standard DP training is insufficient for strong CR. It introduces DP-CERT, a simple, modular framework that weaves augmentation multiplicity, Gaussian input perturbations, regularization, and (optionally) adversarial training into the DPSGD pipeline, enabling CR guarantees without extra network components. Empirically, DP-CERT improves approximate certified accuracy and average certified radius on CIFAR10 under the same DP budget, surpassing prior approaches while maintaining competitive clean accuracy. The work also provides per-sample analyses linking certified radii to local Lipschitz constants and loss-surface smoothness, offering practical guidance for deploying privacy-preserving, robust models in real-world settings.

Abstract

Machine learning models are susceptible to a variety of attacks that can erode trust, including attacks against the privacy of training data, and adversarial examples that jeopardize model accuracy. Differential privacy and certified robustness are effective frameworks for combating these two threats respectively, as they each provide future-proof guarantees. However, we show that standard differentially private model training is insufficient for providing strong certified robustness guarantees. Indeed, combining differential privacy and certified robustness in a single system is non-trivial, leading previous works to introduce complex training schemes that lack flexibility. In this work, we present DP-CERT, a simple and effective method that achieves both privacy and robustness guarantees simultaneously by integrating randomized smoothing into standard differentially private model training. Compared to the leading prior work, DP-CERT gives up to a 2.5% increase in certified accuracy for the same differential privacy guarantee on CIFAR10. Through in-depth per-sample metric analysis, we find that larger certifiable radii correlate with smaller local Lipschitz constants, and show that DP-CERT effectively reduces Lipschitz constants compared to other differentially private training methods. The code is available at github.com/layer6ai-labs/dp-cert.
Paper Structure (44 sections, 14 equations, 19 figures, 4 tables, 2 algorithms)

This paper contains 44 sections, 14 equations, 19 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Present Work.
  3. Preliminaries
  4. Problem Setup.
  5. Differential Privacy and DPSGD.
  6. Certified Robustness.
  7. Randomized Smoothing.
  8. Related Work.
  9. Method
  10. Augmentation Multiplicity.
  11. Adversarial Training.
  12. Regularization.
  13. Summary of Method.
  14. Privacy Guarantee.
  15. Experiment Setup
  16. ...and 29 more sections

Figures (19)

  • Figure 1: The DP-CERT training framework for providing strong CR guarantees within DPSGD.
  • Figure 2: Approximate certified accuracy (ACR) comparison on CIFAR10.
  • Figure 3: Approximate certified accuracy comparison on CIFAR10. The y-axis shows the relative certified accuracy improvement over the strongest baseline method, TransDenoiser, as percentages. Baseline results are from tang2021two.
  • Figure 4: Ablation for consistency regularization, PSAC, and augmentation number on Fashion-MNIST.
  • Figure 5: Per-sample metric comparison, MNIST, $\sigma=0.5$
  • ...and 14 more figures