Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rényi Differential Privacy of the Sampled Gaussian Mechanism

Ilya Mironov, Kunal Talwar, Li Zhang

TL;DR

A numerically stable procedure for precise computation of SGM's Renyi Differential Privacy is described and a nearly tight (within a small constant factor) closed-form bound is proved.

Abstract

The Sampled Gaussian Mechanism (SGM)---a composition of subsampling and the additive Gaussian noise---has been successfully used in a number of machine learning applications. The mechanism's unexpected power is derived from privacy amplification by sampling where the privacy cost of a single evaluation diminishes quadratically, rather than linearly, with the sampling rate. Characterizing the precise privacy properties of SGM motivated development of several relaxations of the notion of differential privacy. This work unifies and fills in gaps in published results on SGM. We describe a numerically stable procedure for precise computation of SGM's Rényi Differential Privacy and prove a nearly tight (within a small constant factor) closed-form bound.

Rényi Differential Privacy of the Sampled Gaussian Mechanism

TL;DR

A numerically stable procedure for precise computation of SGM's Renyi Differential Privacy is described and a nearly tight (within a small constant factor) closed-form bound is proved.

Abstract

The Sampled Gaussian Mechanism (SGM)---a composition of subsampling and the additive Gaussian noise---has been successfully used in a number of machine learning applications. The mechanism's unexpected power is derived from privacy amplification by sampling where the privacy cost of a single evaluation diminishes quadratically, rather than linearly, with the sampling rate. Characterizing the precise privacy properties of SGM motivated development of several relaxations of the notion of differential privacy. This work unifies and fills in gaps in published results on SGM. We describe a numerically stable procedure for precise computation of SGM's Rényi Differential Privacy and prove a nearly tight (within a small constant factor) closed-form bound.

Paper Structure

This paper contains 12 sections, 8 theorems, 59 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction and Definitions
  2. Reducing to a simpler case
  3. RDP Analysis of Single-Dimensional SGM
  4. Proving $A_\alpha \geq B_\alpha$
  5. Closed-Form Bound
  6. Case I: $\alpha<2$
  7. Case IIa: $\alpha\geq 2$ and $q(\alpha-1)<1/3$
  8. Case IIb: $\alpha\geq 2$ and $q(\alpha-1)\geq 1/3$
  9. Numerically Stable Computation
  10. Case I: Integer $\alpha$.
  11. Case II. Fractional $\alpha$.
  12. Discussion

Key Result

Theorem 4

Let $\mathrm{SG}_{q,\sigma}$ be the Sampled Gaussian mechanism for some function $f$. Then $\mathrm{SG}_{q,\sigma}$ satisfies $(\alpha,\varepsilon)$-RDP whenever under the assumption $\|f(S)-f(S')\|_2 \leq 1$ for any adjacent $S,S'\in\mathcal{S}$.

Figures (1)

  • Figure 1: Left: Maximum $\alpha$ as a function of $q$ for $\sigma=4, 10$. For each $\sigma$ four graphs are plotted: bounds due to conditions \ref{['eq:alpha1']} and \ref{['eq:alpha2']}, their minimum, and the exact $\alpha$ so that $\varepsilon=2q^2\alpha/\sigma^2$. Right: RDP of $\mathrm{SG}_{q,\sigma}$ computed exactly (Section \ref{['ss:numerical']}) and bounded according to Theorem \ref{['thm:sgm']}.

Theorems & Definitions (20)

  • Definition 1: Rényi divergence
  • Definition 2: Rényi differential privacy (RDP)
  • Definition 3: Sampled Gaussian Mechanism (SGM)
  • Theorem 4
  • proof
  • Theorem 5
  • proof
  • Lemma 6
  • proof
  • Corollary 7
  • ...and 10 more