Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Privacy-Computation trade-offs in Private Repetition and Metaselection

Kunal Talwar

TL;DR

This work shows strong lower bounds for problems of this kind, showing in particular that for any algorithm that preserves the privacy cost up to a constant factor, the failure probability can only fall polynomially in the computational overhead.

Abstract

A Private Repetition algorithm takes as input a differentially private algorithm with constant success probability and boosts it to one that succeeds with high probability. These algorithms are closely related to private metaselection algorithms that compete with the best of many private algorithms, and private hyperparameter tuning algorithms that compete with the best hyperparameter settings for a private learning algorithm. Existing algorithms for these tasks pay either a large overhead in privacy cost, or a large overhead in computational cost. In this work, we show strong lower bounds for problems of this kind, showing in particular that for any algorithm that preserves the privacy cost up to a constant factor, the failure probability can only fall polynomially in the computational overhead. This is in stark contrast with the non-private setting, where the failure probability falls exponentially in the computational overhead. By carefully combining existing algorithms for metaselection, we prove computation-privacy tradeoffs that nearly match our lower bounds.

Privacy-Computation trade-offs in Private Repetition and Metaselection

TL;DR

This work shows strong lower bounds for problems of this kind, showing in particular that for any algorithm that preserves the privacy cost up to a constant factor, the failure probability can only fall polynomially in the computational overhead.

Abstract

A Private Repetition algorithm takes as input a differentially private algorithm with constant success probability and boosts it to one that succeeds with high probability. These algorithms are closely related to private metaselection algorithms that compete with the best of many private algorithms, and private hyperparameter tuning algorithms that compete with the best hyperparameter settings for a private learning algorithm. Existing algorithms for these tasks pay either a large overhead in privacy cost, or a large overhead in computational cost. In this work, we show strong lower bounds for problems of this kind, showing in particular that for any algorithm that preserves the privacy cost up to a constant factor, the failure probability can only fall polynomially in the computational overhead. This is in stark contrast with the non-private setting, where the failure probability falls exponentially in the computational overhead. By carefully combining existing algorithms for metaselection, we prove computation-privacy tradeoffs that nearly match our lower bounds.

Paper Structure

This paper contains 7 sections, 15 theorems, 18 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Main Lower Bound
  4. Allowing more general meta-algorithms
  5. Lower Bounds for Hyperparameter Tuning
  6. Upper Bounds Trade-offs
  7. Conclusions

Key Result

Theorem 1

(Informal version of thm:main_lb_coded) Let $\mathcal{A}$ be an oracle algorithm that satisfies the following properties: Then for $\delta < O(\frac{\varepsilon\gamma}{\ln T})$, $T \geq \gamma^{\Omega(1/c)}$, or equivalently, $c \geq \Omega(\frac{\ln \frac{1}{\gamma}}{\ln T})$.

Figures (1)

  • Figure 1: Existing and new results on Private Repetition (left) and Private Hyperparameter tuning and Metaselection (right). Green dots show the existing upper bounds $\mathcal{A}_{LT}$ and $\mathcal{A}^T_{\max}$ discussed above, and the gray regions depict the existing excluded regions. The red line shows our new lower bounds: we exclude the full region below the red line. The lower bound nearly matches the blue dashed line that depicts our analysis of hybrid algorithms.

Theorems & Definitions (16)

  • Theorem
  • Theorem
  • Definition 2.1
  • Proposition 1: Extension Lemma
  • Proposition 2
  • Proposition 3
  • Proposition 4
  • Proposition 5
  • Theorem 1
  • Corollary 3.1
  • ...and 6 more