Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Finite de Finetti bounds in relative entropy

Lampros Gavalakis, Oliver Johnson, Ioannis Kontoyiannis

TL;DR

The paper addresses finite de Finetti bounds for exchangeable sequences, linking representations as mixtures of i.i.d. processes to concrete bounds on sampling distances. It develops both total variation and relative-entropy bounds, including two new bounds for abstract spaces derived via Stam's sampling inequality and the empirical-measure mixing measure $\mu_n$; these bounds scale as $\frac{k(k-1)}{2(n-k+1)}$ or are bounded by $\log\left(\frac{n^k (n-k)!}{n!}\right)$, and are essentially independent of alphabet size or space dimension. By connecting de Finetti representations to sampling-without-vs-without replacement, the work sharpens the understanding of how close finite exchangeable vectors are to mixtures of i.i.d. laws. The results have implications for Bayesian modeling, information theory, and probabilistic combinatorics, offering practically tight, dimension-free bounds for a broad class of spaces.

Abstract

We review old and recent finite de Finetti theorems in total variation distance and in relative entropy, and we highlight their connections with bounds on the difference between sampling with and without replacement. We also establish two new finite de Finetti theorems for exchangeable random vectors taking values in arbitrary spaces. These bounds are tight, and they are independent of the size and the dimension of the underlying space.

Finite de Finetti bounds in relative entropy

TL;DR

The paper addresses finite de Finetti bounds for exchangeable sequences, linking representations as mixtures of i.i.d. processes to concrete bounds on sampling distances. It develops both total variation and relative-entropy bounds, including two new bounds for abstract spaces derived via Stam's sampling inequality and the empirical-measure mixing measure ; these bounds scale as or are bounded by , and are essentially independent of alphabet size or space dimension. By connecting de Finetti representations to sampling-without-vs-without replacement, the work sharpens the understanding of how close finite exchangeable vectors are to mixtures of i.i.d. laws. The results have implications for Bayesian modeling, information theory, and probabilistic combinatorics, offering practically tight, dimension-free bounds for a broad class of spaces.

Abstract

We review old and recent finite de Finetti theorems in total variation distance and in relative entropy, and we highlight their connections with bounds on the difference between sampling with and without replacement. We also establish two new finite de Finetti theorems for exchangeable random vectors taking values in arbitrary spaces. These bounds are tight, and they are independent of the size and the dimension of the underlying space.
Paper Structure (8 sections, 13 theorems, 45 equations)

This paper contains 8 sections, 13 theorems, 45 equations.

Table of Contents

  1. Information in probability
  2. de Finetti's representation theorem
  3. Finite de Finetti bounds in total variation
  4. Sampling bounds in total variation
  5. Sampling bounds in relative entropy
  6. Finite de Finetti bounds in relative entropy
  7. New finite de Finetti bounds on abstract spaces
  8. Appendix

Key Result

Theorem 2.1

Let $\{X_n\}$ be an exchangeable process where each $X_n$ takes values in $\{0,1\}$. Then there is a unique Borel probability measure $\mu$ on $[0,1]$ such that, for every $n\geq 1$, where $Q_p(1) = 1- Q_p(0) = p$ is the probability mass function of the Bernoulli distribution with parameter $p$.

Theorems & Definitions (16)

  • Theorem 2.1: de Finetti definetti:31definetti:37
  • Theorem 2.2: Hewitt and Savage hewitt-savage:55
  • Theorem 3.1: Diaconis and Freedman diaconis-freedman:80b
  • Theorem 3.2: Diaconis and Freedman diaconis-freedman:80b
  • Theorem 4.1: Diaconis and Freedman diaconis-freedman:80b
  • Theorem 4.2: Freedman freedman:77
  • Theorem 5.1: Stam stam:78
  • Theorem 6.1: Johnson, Gavalakis and Kontoyiannis gavalakis-olly:arxiv
  • Theorem 6.2: Song, Attiah and Yu yu:24
  • Theorem 6.3: Song, Attiah and Yu yu:24
  • ...and 6 more