Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantifying Information Disclosure During Gradient Descent Using Gradient Uniqueness

Mahmoud Abdelghafar, Maryam Aliakbarpour, Chris Jermaine

TL;DR

This work introduces gradient uniqueness (GNQ) as an attack-agnostic, theoretically grounded metric to quantify information disclosure during mini-batch SGD. GNQ defines a per-example, per-iteration score $GNQ_{ij} = g_{ij}^\top S^{+} g_{ij}$ that captures how much a datapoint's gradient stands out against others, with the total disclosure bounded by a function of the cumulative GNQ. The authors propose a practical GNQ-based defense (data-point censorship via GNQ) and demonstrate through extensive experiments that it can achieve privacy levels comparable to DP-SGD while preserving substantially higher utility, and that GNQ correlates with observed attack success across tasks and SGD configurations. The results suggest GNQ as a valuable tool for privacy auditing, risk-aware data filtering, and informing training-time unlearning decisions, without requiring changes to the underlying SGD algorithm.

Abstract

Disclosing private information via publication of a machine learning model is often a concern. Intuitively, publishing a learned model should be less risky than publishing a dataset. But how much risk is there? In this paper, we present a principled disclosure metric called \emph{gradient uniqueness} that is derived from an upper bound on the amount of information disclosure from publishing a learned model. Gradient uniqueness provides an intuitive way to perform privacy auditing. The mathematical derivation of gradient uniqueness is general, and does not make any assumption on the model architecture, dataset type, or the strategy of an attacker. We examine a simple defense based on monitoring gradient uniqueness, and find that it achieves privacy comparable to classical methods such as DP-SGD, while being substantially better in terms of (utility) testing accuracy.

Quantifying Information Disclosure During Gradient Descent Using Gradient Uniqueness

TL;DR

This work introduces gradient uniqueness (GNQ) as an attack-agnostic, theoretically grounded metric to quantify information disclosure during mini-batch SGD. GNQ defines a per-example, per-iteration score that captures how much a datapoint's gradient stands out against others, with the total disclosure bounded by a function of the cumulative GNQ. The authors propose a practical GNQ-based defense (data-point censorship via GNQ) and demonstrate through extensive experiments that it can achieve privacy levels comparable to DP-SGD while preserving substantially higher utility, and that GNQ correlates with observed attack success across tasks and SGD configurations. The results suggest GNQ as a valuable tool for privacy auditing, risk-aware data filtering, and informing training-time unlearning decisions, without requiring changes to the underlying SGD algorithm.

Abstract

Disclosing private information via publication of a machine learning model is often a concern. Intuitively, publishing a learned model should be less risky than publishing a dataset. But how much risk is there? In this paper, we present a principled disclosure metric called \emph{gradient uniqueness} that is derived from an upper bound on the amount of information disclosure from publishing a learned model. Gradient uniqueness provides an intuitive way to perform privacy auditing. The mathematical derivation of gradient uniqueness is general, and does not make any assumption on the model architecture, dataset type, or the strategy of an attacker. We examine a simple defense based on monitoring gradient uniqueness, and find that it achieves privacy comparable to classical methods such as DP-SGD, while being substantially better in terms of (utility) testing accuracy.

Paper Structure

This paper contains 17 sections, 5 theorems, 38 equations, 9 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Gradient Uniqueness
  4. What Does Gradient Uniqueness Measure?
  5. A GNQ-based defense
  6. Experiments
  7. GNQ-based Defense
  8. Can Gradient Uniqueness Explain What's Observed in Practice?
  9. Conclusion
  10. Gradient Uniqueness as a Measure of Data Privacy
  11. Overview
  12. Problem setup:
  13. Our assumptions:
  14. Gradient Uniqueness as an Upper Bound on Disclosure
  15. Proofs of Theorems and Lemmas
  16. ...and 2 more sections

Key Result

Theorem A.1

In mini-batch SGD (Algorithm alg:new_sgd), for any $j \in [N]$ and corresponding datapoint $d_j \in \mathcal{D}$, the mutual information between its training membership indicator $T_j \in \{0,1\}$ and the estimate $\hat{T}_j \coloneqq \mathcal{F}(\theta_{N_r}, j)$ produced by an attacker $\mathcal{F

Figures (9)

  • Figure 1: GNQ-based privacy auditing.
  • Figure 2: Accuracy of GNQ-based auditing.
  • Figure 3: GNQ vs. attack-based auditing.
  • Figure 4: Image reconstruction after removing the highest-ranked image by GNQ.
  • Figure 5: Attack success rate vs. GNQ.
  • ...and 4 more figures

Theorems & Definitions (9)

  • Theorem A.1
  • Theorem A.1
  • proof
  • Lemma A.2
  • proof
  • Lemma A.3
  • proof
  • Lemma A.4
  • proof