Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A New Bound on the Cumulant Generating Function of Dirichlet Processes

Pierre Perrault, Denis Belomestny, Pierre Ménard, Éric Moulines, Alexey Naumov, Daniil Tiapkin, Michal Valko

TL;DR

This paper addresses bounding the cumulant generating function (CGF) of a Dirichlet process X ~ DP(α ν0) to obtain non-asymptotic concentration bounds for sums of independent DPs. It develops a superadditivity-based approach, connects to Varadhan's integral lemma, and applies Fekete's lemma to derive a CGF bound in terms of the reversed KL divergence. The main result is log M_{DP(α ν0)}(f) ≤ sup_{ν ∈ M1(Ω)} ( Eν[f] − α KL(ν0||ν) ), i.e., the convex conjugate of α KL(ν0||·). The bound yields practical confidence regions and is asymptotically optimal as α → ∞, with demonstrated applicability to problems such as Combinatorial Thompson Sampling.

Abstract

In this paper, we introduce a novel approach for bounding the cumulant generating function (CGF) of a Dirichlet process (DP) $X \sim \text{DP}(αν_0)$, using superadditivity. In particular, our key technical contribution is the demonstration of the superadditivity of $α\mapsto \log \mathbb{E}_{X \sim \text{DP}(αν_0)}[\exp( \mathbb{E}_X[αf])]$, where $\mathbb{E}_X[f] = \int f dX$. This result, combined with Fekete's lemma and Varadhan's integral lemma, converts the known asymptotic large deviation principle into a practical upper bound on the CGF $ \log\mathbb{E}_{X\sim \text{DP}(αν_0)}{\exp(\mathbb{E}_{X}{[f]})} $ for any $α> 0$. The bound is given by the convex conjugate of the scaled reversed Kullback-Leibler divergence $α\mathrm{KL}(ν_0\Vert \cdot)$. This new bound provides particularly effective confidence regions for sums of independent DPs, making it applicable across various fields.

A New Bound on the Cumulant Generating Function of Dirichlet Processes

TL;DR

This paper addresses bounding the cumulant generating function (CGF) of a Dirichlet process X ~ DP(α ν0) to obtain non-asymptotic concentration bounds for sums of independent DPs. It develops a superadditivity-based approach, connects to Varadhan's integral lemma, and applies Fekete's lemma to derive a CGF bound in terms of the reversed KL divergence. The main result is log M_{DP(α ν0)}(f) ≤ sup_{ν ∈ M1(Ω)} ( Eν[f] − α KL(ν0||ν) ), i.e., the convex conjugate of α KL(ν0||·). The bound yields practical confidence regions and is asymptotically optimal as α → ∞, with demonstrated applicability to problems such as Combinatorial Thompson Sampling.

Abstract

In this paper, we introduce a novel approach for bounding the cumulant generating function (CGF) of a Dirichlet process (DP) , using superadditivity. In particular, our key technical contribution is the demonstration of the superadditivity of , where . This result, combined with Fekete's lemma and Varadhan's integral lemma, converts the known asymptotic large deviation principle into a practical upper bound on the CGF for any . The bound is given by the convex conjugate of the scaled reversed Kullback-Leibler divergence . This new bound provides particularly effective confidence regions for sums of independent DPs, making it applicable across various fields.
Paper Structure (6 sections, 8 theorems, 42 equations)

This paper contains 6 sections, 8 theorems, 42 equations.

Table of Contents

  1. Introduction
  2. Large deviation principle
  3. A bound through superadditivity of the CGF
  4. Superadditivity for CGF of DPs
  5. Application to the Combinatorial Thompson Sampling policy
  6. Conclusion

Key Result

Theorem 2.3

$\pa{\DP\pa{\alpha\nu_0}}_\alpha$ satisfies a LDP with speed $\alpha$ and rate function $I(\nu) = \KL{\nu_0}{\nu},$ i.e., for all $B\in \cB\pa{\cM_1(\Omega)}$, if $B^{\mathrm{o}}$ (resp. $\bar{B}$) denotes the interior of $B$ (resp. the closure),

Theorems & Definitions (19)

  • Definition 2.1: Rate function
  • Definition 2.2: Large deviation principle: LDP
  • Theorem 2.3: see ganesh2000large
  • Lemma 2.4: Variational formula for $\Kinf$ honda2015nonasymptoticgarivier2022klucbswitch
  • Corollary 3.1
  • Lemma 3.2: Superadditivity for the DP cumulant-generating function
  • Theorem 3.3: Bound on the cumulant-generating function
  • proof
  • Remark 3.4
  • Remark 3.5
  • ...and 9 more