Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A new tail bound for the sum of bounded independent random variables

Jackson Loper, Jeffrey Regier

TL;DR

The paper tackles tight tail bounds for sums $S=\sum_i X_i$ of independent variables with heterogeneous bounds $X_i\in[0,b_i]$ when the mean $\mu=\mathbb{E}[S]$ is known. It recasts the worst-case Chernoff–Cramér bound as a two-dimensional convex optimization by a minimax reformulation, enabling exact computation of $\varphi^*_{\boldsymbol{b},\mu}(s)$ and yielding the bound $\mathbb{P}(S\ge s) \le \exp(\varphi^*_{\boldsymbol{b},\mu}(s))$, which generalizes Hoeffding and reduces to the specialized bound when $b_i=1$. The authors prove convexity properties, derive a practical two-parameter ( $t,\lambda$ ) formulation, and propose efficient solvers; simulations show the new bound substantially tightens the general Hoeffding bound in applicable tail regimes. This enables tighter hypothesis testing and risk assessment for sums with known mean and heterogeneous bounds, expanding the toolkit beyond the classic, equal-interval Hoeffding bound.

Abstract

We construct a new tail bound for the sum of independent random variables for situations in which the expected value of the sum is known and each random variable lies within a specified interval, which may be different for each variable. This new bound can be computed by solving a two-dimensional convex optimization problem. Simulations demonstrate that the new bound is often substantially tighter than Hoeffding's inequality for cases in which both bounds are applicable.

A new tail bound for the sum of bounded independent random variables

TL;DR

The paper tackles tight tail bounds for sums of independent variables with heterogeneous bounds when the mean is known. It recasts the worst-case Chernoff–Cramér bound as a two-dimensional convex optimization by a minimax reformulation, enabling exact computation of and yielding the bound , which generalizes Hoeffding and reduces to the specialized bound when . The authors prove convexity properties, derive a practical two-parameter ( ) formulation, and propose efficient solvers; simulations show the new bound substantially tightens the general Hoeffding bound in applicable tail regimes. This enables tighter hypothesis testing and risk assessment for sums with known mean and heterogeneous bounds, expanding the toolkit beyond the classic, equal-interval Hoeffding bound.

Abstract

We construct a new tail bound for the sum of independent random variables for situations in which the expected value of the sum is known and each random variable lies within a specified interval, which may be different for each variable. This new bound can be computed by solving a two-dimensional convex optimization problem. Simulations demonstrate that the new bound is often substantially tighter than Hoeffding's inequality for cases in which both bounds are applicable.

Paper Structure

This paper contains 4 sections, 4 theorems, 27 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Notation and preliminaries
  3. Optimizing the worst-case Chernoff--Cramér bound
  4. Simulations

Key Result

Theorem 3.1

Fix $\boldsymbol{b}=(b_1,\ldots b_n)$, $\mu \in [0,\sum_{i=1}^n b_i]$, and $s \in [0,\sum_{i=1}^n b_i]$. Let Then, $\varphi^*_{\boldsymbol{b},\mu}(s)$ from eq:varphistar can be expressed as

Figures (1)

  • Figure 1: The new bounds are tighter than the general Hoeffding inequality. We consider three choices of fixed expected value, $\mu \in \{0.8, 0.9, 0.95\}$. We compare tail bounds based on the general Hoeffding inequality hoeffding_probability_1963 with the new bounds; the latter are calculated using \ref{['thm:lowd']}. The top plot shows the bounds. The bottom plot demonstrates the factor of improvement, showing ratios of the two bounds on a log scale.

Theorems & Definitions (6)

  • Theorem 3.1
  • Theorem 3.2
  • Theorem : Restatement of Theorem 3.1
  • proof
  • Theorem : Restatement of Theorem 3.2
  • proof