Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast-MWEM: Private Data Release in Sublinear Time

Themistoklis Haris, Steve Choi, Mutiraj Laksanawisit

TL;DR

This paper addresses the scalability bottleneck of MWEM by reducing the per-iteration cost of the Exponential Mechanism from $Θ(m)$ to $Θ(√m)$ in expectation. The key idea is Lazy Gumbel Sampling, which leverages top-$k$ score retrieval via a $k$-MIPS index to avoid exhaustive scoring, while maintaining differential privacy through standard DP composition. The authors instantiate Fast-MWEM for private linear query release and privately solving Linear Programs, achieving sublinear runtime in the number of queries/constraints and preserving utility guarantees. Empirical results on synthetic benchmarks confirm substantial runtime improvements with competitive accuracy, particularly when using advanced indices like HNSW or IVF. This work enhances the practicality of private data release and private optimization in high-dimensional settings, enabling scalable private analysis.

Abstract

The Multiplicative Weights Exponential Mechanism (MWEM) is a fundamental iterative framework for private data analysis, with broad applications such as answering $m$ linear queries, or privately solving systems of $m$ linear constraints. However, a critical bottleneck hindering its scalability is the $Θ(m)$ time complexity required to execute the exponential mechanism in each iteration. We introduce a modification to the MWEM framework that improves the per-iteration runtime dependency to $Θ(\sqrt{m})$ in expectation. This is done via a lazy sampling approach to the Report-Noisy-Max mechanism, which we implement efficiently using Gumbel noise and a $k$-Nearest Neighbor data structure. This allows for the rapid selection of the approximate score in the exponential mechanism without an exhaustive linear scan. We apply our accelerated framework to the problems of private linear query release and solving Linear Programs (LPs) under neighboring constraint conditions and low-sensitivity assumptions. Experimental evaluation confirms that our method provides a substantial runtime improvement over classic MWEM.

Fast-MWEM: Private Data Release in Sublinear Time

TL;DR

This paper addresses the scalability bottleneck of MWEM by reducing the per-iteration cost of the Exponential Mechanism from to in expectation. The key idea is Lazy Gumbel Sampling, which leverages top- score retrieval via a -MIPS index to avoid exhaustive scoring, while maintaining differential privacy through standard DP composition. The authors instantiate Fast-MWEM for private linear query release and privately solving Linear Programs, achieving sublinear runtime in the number of queries/constraints and preserving utility guarantees. Empirical results on synthetic benchmarks confirm substantial runtime improvements with competitive accuracy, particularly when using advanced indices like HNSW or IVF. This work enhances the practicality of private data release and private optimization in high-dimensional settings, enabling scalable private analysis.

Abstract

The Multiplicative Weights Exponential Mechanism (MWEM) is a fundamental iterative framework for private data analysis, with broad applications such as answering linear queries, or privately solving systems of linear constraints. However, a critical bottleneck hindering its scalability is the time complexity required to execute the exponential mechanism in each iteration. We introduce a modification to the MWEM framework that improves the per-iteration runtime dependency to in expectation. This is done via a lazy sampling approach to the Report-Noisy-Max mechanism, which we implement efficiently using Gumbel noise and a -Nearest Neighbor data structure. This allows for the rapid selection of the approximate score in the exponential mechanism without an exhaustive linear scan. We apply our accelerated framework to the problems of private linear query release and solving Linear Programs (LPs) under neighboring constraint conditions and low-sensitivity assumptions. Experimental evaluation confirms that our method provides a substantial runtime improvement over classic MWEM.
Paper Structure (40 sections, 18 theorems, 70 equations, 9 figures, 5 algorithms)

This paper contains 40 sections, 18 theorems, 70 equations, 9 figures, 5 algorithms.

Table of Contents

  1. Introduction
  2. Our Results and Techniques
  3. Private Linear Query Release.
  4. Private Linear Programming.
  5. Our Techniques.
  6. Experimental Evaluation.
  7. Related Work
  8. Preliminaries
  9. Method & Framework
  10. Setting
  11. The MWEM algorithm
  12. "Lazy" Exponential Mechanism
  13. Fast MWEM with a perfect index
  14. Imperfect indices
  15. Preserving runtime
  16. ...and 25 more sections

Key Result

Theorem 2.3

The Exponential Mechanism is $\varepsilon$-DP and runs in $O(|R|)$ time. Utility-wise, if EM outputs some candidate $\widehat{h} \in R$, then: where $q_{\max} = \max\limits_{i \in R} s(D,h_i)$ is the maximum score of any candidate.

Figures (9)

  • Figure 1: Observed speed-up factor of Fast-MWEM on linear queries over exhaustive search for IVF and HNSW indices.
  • Figure 2: The error difference between MWEM and Fast MWEM
  • Figure 3: Error over iterations for different indices
  • Figure 4: The performance of Fast MWEM for different indices.
  • Figure 5: Violated constraints across different indices.
  • ...and 4 more figures

Theorems & Definitions (43)

  • Definition 2.1
  • Definition 2.2
  • Theorem 2.3
  • Theorem 3.1
  • Lemma 3.2
  • Theorem 3.3
  • proof
  • Definition 3.4
  • Theorem 4.1
  • Definition 4.2
  • ...and 33 more