Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PrivacyGuard: A Modular Framework for Privacy Auditing in Machine Learning

Luca Melis, Matthew Grange, Iden Kalemaj, Karan Chadha, Shengyuan Hu, Elena Kashtelyan, Will Bullock

TL;DR

PrivacyGuard addresses the gap between theoretical privacy guarantees and practical privacy risk assessment by delivering a modular, PyTorch‑based framework for empirical privacy analysis. It combines state‑of‑the‑art membership inference attacks (LiRA and RMIA), robust auditing methods including bootstrap and $f$‑DP auditing, and probabilistic extraction analyses for language models, enabling end‑to‑end privacy evaluation across supervised and generative settings. Key contributions include a clean modular architecture with attack modules and analysis nodes, concrete demonstrations on CIFAR‑10 and Enron, and practical tooling for empirical privacy lower bounds via $\epsilon$ and $f$‑DP curves. The framework emphasizes extensibility, rigorous testing, and tutorials, offering a practical, open‑source resource for researchers and practitioners to quantify privacy risks and guide responsible deployment of AI systems. PrivacyGuard thus provides a scalable, attack‑driven approach to empirical privacy auditing with applicability to both traditional and large language model workflows.

Abstract

The increasing deployment of Machine Learning (ML) models in sensitive domains motivates the need for robust, practical privacy assessment tools. PrivacyGuard is a comprehensive tool for empirical differential privacy (DP) analysis, designed to evaluate privacy risks in ML models through state-of-the-art inference attacks and advanced privacy measurement techniques. To this end, PrivacyGuard implements a diverse suite of privacy attack -- including membership inference , extraction, and reconstruction attacks -- enabling both off-the-shelf and highly configurable privacy analyses. Its modular architecture allows for the seamless integration of new attacks, and privacy metrics, supporting rapid adaptation to emerging research advances. We make PrivacyGuard available at https://github.com/facebookresearch/PrivacyGuard.

PrivacyGuard: A Modular Framework for Privacy Auditing in Machine Learning

TL;DR

PrivacyGuard addresses the gap between theoretical privacy guarantees and practical privacy risk assessment by delivering a modular, PyTorch‑based framework for empirical privacy analysis. It combines state‑of‑the‑art membership inference attacks (LiRA and RMIA), robust auditing methods including bootstrap and ‑DP auditing, and probabilistic extraction analyses for language models, enabling end‑to‑end privacy evaluation across supervised and generative settings. Key contributions include a clean modular architecture with attack modules and analysis nodes, concrete demonstrations on CIFAR‑10 and Enron, and practical tooling for empirical privacy lower bounds via and ‑DP curves. The framework emphasizes extensibility, rigorous testing, and tutorials, offering a practical, open‑source resource for researchers and practitioners to quantify privacy risks and guide responsible deployment of AI systems. PrivacyGuard thus provides a scalable, attack‑driven approach to empirical privacy auditing with applicability to both traditional and large language model workflows.

Abstract

The increasing deployment of Machine Learning (ML) models in sensitive domains motivates the need for robust, practical privacy assessment tools. PrivacyGuard is a comprehensive tool for empirical differential privacy (DP) analysis, designed to evaluate privacy risks in ML models through state-of-the-art inference attacks and advanced privacy measurement techniques. To this end, PrivacyGuard implements a diverse suite of privacy attack -- including membership inference , extraction, and reconstruction attacks -- enabling both off-the-shelf and highly configurable privacy analyses. Its modular architecture allows for the seamless integration of new attacks, and privacy metrics, supporting rapid adaptation to emerging research advances. We make PrivacyGuard available at https://github.com/facebookresearch/PrivacyGuard.

Paper Structure

This paper contains 24 sections, 11 equations, 8 figures, 4 tables.

Table of Contents

  1. Background and Introduction
  2. Design Principles and Features
  3. Privacy Attack Module
  4. Analysis Nodes
  5. Robustness and Testing Framework
  6. Auditing methodology and Results
  7. Membership Inference Attacks on ML Models
  8. Likelihood Ratio Attack.
  9. Robust Membership Inference Attack.
  10. Auditing Analyses
  11. Bootstrap-based analysis.
  12. $f$-DP auditing mahloujifar2025auditing.
  13. Probabilistic Extraction on Language Models
  14. Notation.
  15. The matching function $\texttt{match}(\cdot,\cdot)$ can be set as:
  16. ...and 9 more sections

Figures (8)

  • Figure 1: PrivacyGuard Attack module
  • Figure 2: PrivacyGuard Analysis Node component
  • Figure 3: Histogram of per-example scores computed by adversary for likelihood ratio attack (LiRA).
  • Figure 4: ROC curve of adversary, zooming in at low false positive rates. Training members are the "positive class"
  • Figure 5: ROC curve of adversary for RMIA, zooming in at low false positive rates. Training members are the "positive class". The parameter $\alpha$ is the gap in predictions for a datapoint between a model including and excluding that datapoint.
  • ...and 3 more figures

Theorems & Definitions (3)

  • definition 1: DMNS06
  • definition 2: Discoverable Extraction
  • definition 3: $(n,p)$-discoverable extraction