An extension of McDiarmid's inequality
Richard Combes
TL;DR
This work generalizes McDiarmid's inequality to functions whose bounded-differences property holds only on a high-probability subset ${\cal Y}$, showing that such functions concentrate around the conditional expectation $\mathbb{E}[f(X)\mid X\in{\cal Y}]$ with a bound that includes an additive term involving $p=\mathbb{P}[X\notin{\cal Y}]$ and $\bar c=\sum_i c_i$. The authors prove the main result via a Lipschitz extension (McShane) to construct a 1-Lipschitz surrogate $g$ and bound $\mathbb{E}[g(X)]$ in terms of $\mathbb{E}[f(X)\mid X\in{\cal Y}]$ plus $p\bar c$, yielding a concentration inequality of the form $\mathbb{P}\left[ f(X) - \mathbb{E}[f(X)\mid X\in{\cal Y}] \ge \varepsilon + p\bar c \right] \le p + \exp\left(-\dfrac{2\varepsilon^2}{\sum c_i^2}\right)$. The approach extends to general metric spaces via Wasserstein distances, linking concentration of Lipschitz functions to the coupling between conditional distributions and providing concrete corollaries for Gaussian and finite-diameter spaces. The paper demonstrates the utility with applications to triangles in random graphs, concentration of maximum likelihood estimators, and empirical risk minimization, delivering significantly tighter bounds than classical McDiarmid bounds when the bounded-differences condition holds with high probability. Overall, the results offer a flexible framework for sharp concentration in settings where the bounded-differences property is only approximately guaranteed and extend naturally to non-Euclidean domains.
Abstract
We generalize McDiarmid's inequality for functions with bounded differences on a high probability set, using an extension argument. Those functions concentrate around their conditional expectations. We further extend the results to concentration in general metric spaces.