A Third Information-Theoretic Approach to Finite de Finetti Theorems
Mario Berta, Lampros Gavalakis, Ioannis Kontoyiannis
TL;DR
The paper addresses finite de Finetti representations for exchangeable vectors by developing a new information-theoretic approach based on mutual information, inspired by quantum information theory. It proves a general bound: for exchangeable $X_1^n$, there exists a mixing measure $mu$ such that $D(P_{X_1^k} olinebreak o M_{k,mu}) olinebreak ext{ bounded by } rac{1}{n-k+1} extstyle imes extstyle igl( olinebreak extstyle extstyle extstyle extstyle ext{ } igr)$, specifically $D(P_{X_1^k} olinebreak o M_{k,mu}) olinebreak ext{ ≤ } rac{1}{n-k+1} extstyle extstyle extstyle ext{ } igl( extstyle extstyle extstyle extstyle I(X_1^{i-1};X_k^n)igr)$. Specializing to finite alphabets yields a concrete corollary $D(P_{X_1^k} olinebreak o M_{k,mu}) olinebreak ext{ ≤ } rac{k(k-1)}{2(n-k-1)} ext{ } ext{ log }|A|$, and the paper also provides an explicit discrete-alphabet bound and two illustrative examples. A Stam-based, non-information-theoretic bound is given as well. Overall, the results improve the understanding of finite de Finetti representations in terms of relative entropy, extend to general spaces, and connect to quantum-information methods.
Abstract
A new finite form of de Finetti's representation theorem is established using elementary information-theoretic tools. The distribution of the first $k$ random variables in an exchangeable vector of $n\geq k$ random variables is close to a mixture of product distributions. Closeness is measured in terms of the relative entropy and an explicit bound is provided. This bound is tighter than those obtained via earlier information-theoretic proofs, and its utility extends to random variables taking values in general spaces. The core argument employed has its origins in the quantum information-theoretic literature.