Tight Non-asymptotic Inference via Sub-Gaussian Intrinsic Moment Norm

TL;DR

This work addresses tight non-asymptotic inference under sub-Gaussian data by introducing the intrinsic moment norm $\|X\|_G$, which tightly approximates MGF bounds and enables sharper concentration than traditional variance proxies. It develops practical estimation methods, including a plug-in estimator and robust MOM-based estimators, along with small-sample techniques such as leave-one-out Hodges-Lehmann methods, all coordinated through the novel sub-Gaussian plot to diagnose sub-Gaussianity. The authors extend these tools to reinforcement learning, notably a BeUCB algorithm for multi-armed bandits with provable regret bounds that match minimax rates up to constants. Overall, the approach provides a robust, estimable, and theoretically sharp framework for non-asymptotic uncertainty quantification in sub-Gaussian settings with concrete applications to bandits and RL.

Abstract

In non-asymptotic learning, variance-type parameters of sub-Gaussian distributions are of paramount importance. However, directly estimating these parameters using the empirical moment generating function (MGF) is infeasible. To address this, we suggest using the sub-Gaussian intrinsic moment norm [Buldygin and Kozachenko (2000), Theorem 1.3] achieved by maximizing a sequence of normalized moments. Significantly, the suggested norm can not only reconstruct the exponential moment bounds of MGFs but also provide tighter sub-Gaussian concentration inequalities. In practice, we provide an intuitive method for assessing whether data with a finite sample size is sub-Gaussian, utilizing the sub-Gaussian plot. The intrinsic moment norm can be robustly estimated via a simple plug-in approach. Our theoretical findings are also applicable to reinforcement learning, including the multi-armed bandit scenario.

Figures (11)

• Figure 1: CIs via Hoeffding's inequality (red line) and B-E-corrected CLT (blue line). This illustration points out the deficiencies of the B-E-corrected CLT under small sample cases. Interestingly, it suggests that a straightforward application of Hoeffding's inequality might even offer better performance.
• Figure 1: Sub-Gaussian plot for standard Gaussian and standard exponential random variables with $n = 1000$. (Left: The two dotted lines denote a triangular region where the points are likely to fall with high probability. The data points corresponding to the exponential random variable predominantly occupy a curved triangular region, thereby exhibiting quadratic behavior.)
• Figure 1: OP refers to the plug-in estimator for the optimal variance proxy, which employs the computational trick detailed in lieber2022estimating. DE represents the naive plug-in estimator as defined in equation \ref{['DE']}, while MOM denotes the MOM estimator given in equation \ref{['MOM']} with a fixed block size of $b = 5$. MOM_LOCV is the MOM estimator where the block number $b$ is selected using the leave-one-out method, as outlined in equation \ref{['MOM_LOOCV']}. The absolute error vector, averaged over $n \in [250]$, for the estimators (OP, DE, MOM, MOM_LOCV) is $(1.54, 0.07, 0.06, 0.05)$, $(1.30, 0.25, 0.07, 0.07)$, $(1.49, 0.00, 0.00, 0.00)$, and $(1.78, 0.59, 0.00, 0.00)$, respectively. These vectors correspond to the four different figures, arranged from top to bottom.
• Figure 1: The regret of MAB with sub-G rewards under three methods. $x$-axis represents the round number and $y$-axis is the cumulative regret.
• Figure : Bernoulli

Theorems & Definitions (17)

• Definition 1.1
• Definition 1.2: Page 6 in buldygin2000metric
• Definition 2.1: Lower intrinsic moment norm
• Theorem 2.2
• Lemma 2.3: A reverse Chernoff inequality
• Lemma 3.1
• Example 1
• Theorem 3.2
• Proposition 4.1
• Theorem 4.2: Finite sample guaranteed coverage for MOM estimted norms