Perturbed Kullback-Leibler Deviation Bounds for Dirichlet Processes

TL;DR

The paper develops perturbed Kullback-Leibler deviation bounds for Dirichlet processes to achieve sharper, non-asymptotic concentration results. It introduces a controlled perturbation of the base measure to tighten KL-type tail bounds and presents two complementary proof strategies: a superadditivity-based argument via Fekete’s lemma and a reduction to the Beta distribution, building a unified treatment that extends Beta-bound inequalities to DP settings. The work connects DP concentration to large deviation theory by leveraging the DP variational KL formula $\, ext{KL}_{\\inf}$ and the Gamma-process DP representation, and it discusses implications for non-asymptotic deviations and potential extensions beyond unit-mass perturbations. Overall, the results provide tighter, non-asymptotic probabilistic guarantees for DP tails, with theoretical insights relevant to Bayesian nonparametrics and probabilistic bounds in high-dimensional DP contexts.

Abstract

We present new and improved non-asymptotic deviation bounds for Dirichlet processes (DPs), formulated using the Kullback-Leibler (KL) divergence, which is known for its optimal characterization of the asymptotic behavior of DPs. Our method involves incorporating a controlled perturbation within the KL bound, effectively shifting the base distribution of the DP in the upper bound. Our proofs rely on two independent approaches. In the first, we use superadditivity techniques to convert asymptotic bounds into non-asymptotic ones via Fekete's lemma. In the second, we carefully reduce the problem to the Beta distribution case. Some of our results extend similar inequalities derived for the Beta distribution, as presented in [27].