Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Statistical Complexity of Sample Amplification

Brian Axelrod, Shivam Garg, Yanjun Han, Vatsal Sharan, Gregory Valiant

TL;DR

This work provides a firm statistical foundation for the sample amplification problem by deriving generally applicable amplification procedures, lower bound techniques and connections to existing statistical notions.

Abstract

The ``sample amplification'' problem formalizes the following question: Given $n$ i.i.d. samples drawn from an unknown distribution $P$, when is it possible to produce a larger set of $n+m$ samples which cannot be distinguished from $n+m$ i.i.d. samples drawn from $P$? In this work, we provide a firm statistical foundation for this problem by deriving generally applicable amplification procedures, lower bound techniques and connections to existing statistical notions. Our techniques apply to a large class of distributions including the exponential family, and establish a rigorous connection between sample amplification and distribution learning.

On the Statistical Complexity of Sample Amplification

TL;DR

This work provides a firm statistical foundation for the sample amplification problem by deriving generally applicable amplification procedures, lower bound techniques and connections to existing statistical notions.

Abstract

The ``sample amplification'' problem formalizes the following question: Given i.i.d. samples drawn from an unknown distribution , when is it possible to produce a larger set of samples which cannot be distinguished from i.i.d. samples drawn from ? In this work, we provide a firm statistical foundation for this problem by deriving generally applicable amplification procedures, lower bound techniques and connections to existing statistical notions. Our techniques apply to a large class of distributions including the exponential family, and establish a rigorous connection between sample amplification and distribution learning.
Paper Structure (48 sections, 30 theorems, 245 equations, 1 figure, 1 table, 3 algorithms)

This paper contains 48 sections, 30 theorems, 245 equations, 1 figure, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Connections, limitations and future work
  3. Preliminaries
  4. Sample amplification via sufficient statistics
  5. The general idea
  6. Computational issues
  7. General theory for exponential families
  8. Sample amplification via learning
  9. The general result
  10. The shuffling approach
  11. Minimax lower bounds
  12. General idea
  13. Product models
  14. Discussions on sample amplification versus learning
  15. An example where the complexity of sample amplification is $o(\sqrt{d})$
  16. ...and 33 more sections

Key Result

Lemma 4.4

If the log-partition function $A(\theta)$ satisfies then the exponential family satisfies the moment condition $\mathsf{M}_k$ for all $k\in \mathbb{N}$. Here for a $k$-tensor $T$ and vectors $u_1,\cdots,u_k$, $T[u_1;\cdots;u_k]$ denotes the value of $\langle T, u_1\otimes \cdots \otimes u_k \rangle$.

Figures (1)

  • Figure 1: Sample amplification experiments. The $x$-axis corresponds to the amount of amplification, $m$, and the shaded area depicts the 95% confidence interval based on 5,000 simulations.

Theorems & Definitions (52)

  • Definition 1.1: Sample Amplification
  • Definition 3.1: Le Cam's distance; see le1972limitsLeCam1986asymptoticle1990asymptotics
  • Example 4.1: Gaussian location model with known covariance
  • Example 4.2: Computation in Gaussian location model
  • Definition 4.3: Exponential family
  • Lemma 4.4
  • Theorem 4.5
  • Theorem 4.6
  • Definition 5.1: $\chi^2$-estimation error
  • Theorem 5.2
  • ...and 42 more