Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Weighted Fourier Factorizations: Optimal Gaussian Noise for Differentially Private Marginal and Product Queries

Christian Janos Lebeda, Aleksandar Nikolov, Haohua Tang

TL;DR

The paper investigates privately releasing marginal queries under differential privacy using a correlated Gaussian noise approach in the Fourier domain. It introduces a weighted marginal workload framework and a Fourier based factorization mechanism that is exactly optimal among factorization mechanisms, with polynomial running time. The authors extend the framework to product queries and extended marginals, and provide matching upper and lower bounds to establish near optimality. The results enable efficient, provably optimal estimation for high dimensional, weighted marginal workloads with practical implications for statistics and official statistics releases.

Abstract

We revisit the task of releasing marginal queries under differential privacy with additive (correlated) Gaussian noise. We first give a construction for answering arbitrary workloads of weighted marginal queries, over arbitrary domains. Our technique is based on releasing queries in the Fourier basis with independent noise with carefully calibrated variances, and reconstructing the marginal query answers using the inverse Fourier transform. We show that our algorithm, which is a factorization mechanism, is exactly optimal among all factorization mechanisms, both for minimizing the sum of weighted noise variances, and for minimizing the maximum noise variance. Unlike algorithms based on optimizing over all factorization mechanisms via semidefinite programming, our mechanism runs in time polynomial in the dataset and the output size. This construction recovers results of Xiao et al. [Neurips 2023] with a simpler algorithm and optimality proof, and a better running time. We then extend our approach to a generalization of marginals which we refer to as product queries. We show that our algorithm is still exactly optimal for this more general class of queries. Finally, we show how to embed extended marginal queries, which allow using a threshold predicate on numerical attributes, into product queries. We show that our mechanism is almost optimal among all factorization mechanisms for extended marginals, in the sense that it achieves the optimal (maximum or average) noise variance up to lower order terms.

Weighted Fourier Factorizations: Optimal Gaussian Noise for Differentially Private Marginal and Product Queries

TL;DR

The paper investigates privately releasing marginal queries under differential privacy using a correlated Gaussian noise approach in the Fourier domain. It introduces a weighted marginal workload framework and a Fourier based factorization mechanism that is exactly optimal among factorization mechanisms, with polynomial running time. The authors extend the framework to product queries and extended marginals, and provide matching upper and lower bounds to establish near optimality. The results enable efficient, provably optimal estimation for high dimensional, weighted marginal workloads with practical implications for statistics and official statistics releases.

Abstract

We revisit the task of releasing marginal queries under differential privacy with additive (correlated) Gaussian noise. We first give a construction for answering arbitrary workloads of weighted marginal queries, over arbitrary domains. Our technique is based on releasing queries in the Fourier basis with independent noise with carefully calibrated variances, and reconstructing the marginal query answers using the inverse Fourier transform. We show that our algorithm, which is a factorization mechanism, is exactly optimal among all factorization mechanisms, both for minimizing the sum of weighted noise variances, and for minimizing the maximum noise variance. Unlike algorithms based on optimizing over all factorization mechanisms via semidefinite programming, our mechanism runs in time polynomial in the dataset and the output size. This construction recovers results of Xiao et al. [Neurips 2023] with a simpler algorithm and optimality proof, and a better running time. We then extend our approach to a generalization of marginals which we refer to as product queries. We show that our algorithm is still exactly optimal for this more general class of queries. Finally, we show how to embed extended marginal queries, which allow using a threshold predicate on numerical attributes, into product queries. We show that our mechanism is almost optimal among all factorization mechanisms for extended marginals, in the sense that it achieves the optimal (maximum or average) noise variance up to lower order terms.
Paper Structure (27 sections, 35 theorems, 159 equations, 2 figures, 3 algorithms)

This paper contains 27 sections, 35 theorems, 159 equations, 2 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Problem Setup
  3. Our Contributions, and Factorization Mechanisms
  4. Weighted marginal workloads.
  5. Optimality among Factorization Mechanisms.
  6. Extensions.
  7. Comparison with ResidualPlanner.
  8. Technical Intuition
  9. Preliminaries
  10. Marginal Queries
  11. Extensions
  12. Differential Privacy
  13. Optimal Factorization for Weighted Marginal Queries
  14. Marginal Queries in the Fourier Basis
  15. Warm-up: Estimating All $k$-way Marginals
  16. ...and 12 more sections

Key Result

Lemma 2.3

Let $q \colon \mathcal{U}^* \rightarrow \mathbb{R}^d$ be a set of queries with $\ell_2$ sensitivity $\Delta q \coloneq \max_{D \sim D'} \|q(D) - q(D')\|_2$. Then the mechanism that outputs $q(X) + Z$ where $Z \sim \mathcal{N}\left(0, \frac{(\Delta q)^2}{\mu^2} I_d\right)$ satisfies $\mu$-GDP.

Figures (2)

  • Figure 1: Comparison for small values of $k$, $m$, and $d$ of the standard deviation from \ref{['thm:upper-bound']} relative to the baseline that adds i.i.d. noise with magnitude $\sqrt{d \choose k}$. When $d$ increases the improvement ratio approaches $(1-1/m)^k$.
  • Figure 2: The functions $\eta(m)$ and $\zeta(m)$, as well as their difference $\eta(m)-\zeta(m)$.

Theorems & Definitions (69)

  • Definition 2.1: Neighboring datasets
  • Definition 2.2: Gaussian Differential Privacy DongRS22-GDP
  • Lemma 2.3: The Gaussian mechanism
  • Lemma 2.4: Gaussian noise for marginal queries
  • proof
  • Lemma 2.5: Post-processing DongRS22-GDP
  • Lemma 2.6: Composition DongRS22-GDP
  • Definition 2.7: Roots of unity
  • Lemma 2.8: Multidimensional fast Fourier transform cooley1965algorithm
  • Definition 2.9: Complex Gaussian distribution
  • ...and 59 more