Concentration Inequalities for Statistical Inference
Huiming Zhang, Song Xi Chen
TL;DR
This paper surveys non-asymptotic concentration inequalities across distribution-free, sub-Gaussian, sub-exponential, sub-Gamma, and sub-Weibull classes, highlighting how each tail class yields precise probabilistic bounds for sums, maxima, and empirical processes. It connects foundational bounds (e.g., Markov, Chebyshev, Chernoff, Hoeffding, McDiarmid, Azuma) to modern high-dimensional statistics, presenting constant-specified results and extending techniques to random matrices and Poisson models. Key contributions include sharpened constants for quadratic forms (Hanson–Wright), refined Bernstein-type bounds via sub-Gamma and sub-Weibull frameworks, and non-asymptotic oracle inequalities for penalized regressions in HD settings. The work emphasizes practical finite-sample guarantees for HD linear and Poisson regression, U-statistics, and empirical processes, with implications for reliable inference, model selection, and hypothesis testing under heavy tails and dependence. Overall, it provides a cohesive toolkit for non-asymptotic statistical inference in contemporary high-dimensional problems, bridging theory with applicable constants and methods.
Abstract
This paper gives a review of concentration inequalities which are widely employed in non-asymptotical analyses of mathematical statistics in a wide range of settings, from distribution-free to distribution-dependent, from sub-Gaussian to sub-exponential, sub-Gamma, and sub-Weibull random variables, and from the mean to the maximum concentration. This review provides results in these settings with some fresh new results. Given the increasing popularity of high-dimensional data and inference, results in the context of high-dimensional linear and Poisson regressions are also provided. We aim to illustrate the concentration inequalities with known constants and to improve existing bounds with sharper constants.