Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey

Lea Demelius, Roman Kern, Andreas Trügler

TL;DR

This paper systematically surveys centralized differential privacy in deep learning from 2019 to 2023, addressing the privacy-utility trade-off, auditing/evaluation, and threats beyond membership/inference, as well as private generative models and domain-specific applications. It synthesizes core methods such as DP-SGD and PATE, reviews privately trained generative approaches (autoencoders and GANs), and examines a broad range of applications (medical imaging, face recognition, NLP, and more). Key findings include persistent gaps between theoretical privacy guarantees and empirical attacks, the importance of realistic benchmarks, and the potential of combining techniques to improve utility under DP, while stressing careful interpretation of synthetic data and the need for standardized taxonomy. The work provides a structured, theory-to-practice guide for researchers and practitioners aiming to deploy DP-DL in real-world settings, highlighting both progress and open challenges in privacy-preserving centralized deep learning.

Abstract

Differential Privacy has become a widely popular method for data protection in machine learning, especially since it allows formulating strict mathematical privacy guarantees. This survey provides an overview of the state-of-the-art of differentially private centralized deep learning, thorough analyses of recent advances and open problems, as well as a discussion of potential future developments in the field. Based on a systematic literature review, the following topics are addressed: auditing and evaluation methods for private models, improvements of privacy-utility trade-offs, protection against a broad range of threats and attacks, differentially private generative models, and emerging application domains.

Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey

TL;DR

This paper systematically surveys centralized differential privacy in deep learning from 2019 to 2023, addressing the privacy-utility trade-off, auditing/evaluation, and threats beyond membership/inference, as well as private generative models and domain-specific applications. It synthesizes core methods such as DP-SGD and PATE, reviews privately trained generative approaches (autoencoders and GANs), and examines a broad range of applications (medical imaging, face recognition, NLP, and more). Key findings include persistent gaps between theoretical privacy guarantees and empirical attacks, the importance of realistic benchmarks, and the potential of combining techniques to improve utility under DP, while stressing careful interpretation of synthetic data and the need for standardized taxonomy. The work provides a structured, theory-to-practice guide for researchers and practitioners aiming to deploy DP-DL in real-world settings, highlighting both progress and open challenges in privacy-preserving centralized deep learning.

Abstract

Differential Privacy has become a widely popular method for data protection in machine learning, especially since it allows formulating strict mathematical privacy guarantees. This survey provides an overview of the state-of-the-art of differentially private centralized deep learning, thorough analyses of recent advances and open problems, as well as a discussion of potential future developments in the field. Based on a systematic literature review, the following topics are addressed: auditing and evaluation methods for private models, improvements of privacy-utility trade-offs, protection against a broad range of threats and attacks, differentially private generative models, and emerging application domains.
Paper Structure (27 sections, 4 equations, 2 figures, 8 tables)

This paper contains 27 sections, 4 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Contributions of this survey
  3. Survey methodology
  4. Preliminaries
  5. Differential Privacy (DP)
  6. Deep neural networks and stochastic gradient descent (SGD)
  7. Privacy threats for deep learning models
  8. DP algorithms for deep learning
  9. Auditing and Evaluation
  10. Attack-based Evaluation
  11. Evaluation of subgroups: bias and fairness
  12. Improving the privacy-utility trade-off of DP-DL
  13. Beyond membership and attribute inference: Applying DP-DL to protect against other threats
  14. Differentially private generative models
  15. Autoencoders
  16. ...and 12 more sections

Figures (2)

  • Figure 1: Flow diagram of the paper selection process.
  • Figure 2: Examples of tempered sigmoid functions in comparison with the ReLU function. Tempered sigmoid functions with their parameters $s$, $T$ and $o$ are bounded activation functions proposed by Papernot et al. papernot_tempered_2020 to improve private deep learning. ReLU is an unbounded activation function commonly used in deep learning.