Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Growth-Optimal E-Variables and an extension to the multivariate Csiszár-Sanov-Chernoff Theorem

Peter Grünwald, Yunda Hao, Akshay Balsubramani

TL;DR

The paper investigates growth-rate-optimal e-variables (GROW) for a simple multivariate null against composite alternatives and connects these to Csiszár-Sanov-Chernoff (CSC) concentration bounds. It leverages the exponential-family structure and the Pythagorean property to derive exact and bound-form GROW e-variables, first for convex mean-sets M_1 and then for surrounding, nonconvex M_1, introducing relative GROW and minimax regret as robust alternatives. For d = 1, a complete GROW characterization is given and extended to a CSC bound in the surrounding setting, while for general d, a relative GROW bound with a new CSC-type result is established, including asymptotic growth-rate expressions GROW_n = n underline{D} − (d−1)/2 log n + O(1). The work highlights the role of boundary geometry, KL-behavior, and information-theoretic constructs (e.g., Shtarkov, NML) in designing powerful, anytime-valid tests and provides a roadmap for future extensions to higher dimensions and refined asymptotics. Overall, the paper contributes a unified information-theoretic framework linking e-values, CSC bounds, and asymptotic growth, with potential impact on sequential testing and model-selection contexts that require robust, highly efficient testing under optional stopping.

Abstract

We consider growth-optimal e-variables with maximal e-power, both in an absolute and relative sense, for simple null hypotheses for a $d$-dimensional random vector, and multivariate composite alternatives represented as a set of $d$-dimensional means $\meanspace_1$. These include, among others, the set of all distributions with mean in $\meanspace_1$, and the exponential family generated by the null restricted to means in $\meanspace_1$. We show how these optimal e-variables are related to Csiszár-Sanov-Chernoff bounds, first for the case that $\meanspace_1$ is convex (these results are not new; we merely reformulate them) and then for the case that $\meanspace_1$ `surrounds' the null hypothesis (these results are new).

Growth-Optimal E-Variables and an extension to the multivariate Csiszár-Sanov-Chernoff Theorem

TL;DR

The paper investigates growth-rate-optimal e-variables (GROW) for a simple multivariate null against composite alternatives and connects these to Csiszár-Sanov-Chernoff (CSC) concentration bounds. It leverages the exponential-family structure and the Pythagorean property to derive exact and bound-form GROW e-variables, first for convex mean-sets M_1 and then for surrounding, nonconvex M_1, introducing relative GROW and minimax regret as robust alternatives. For d = 1, a complete GROW characterization is given and extended to a CSC bound in the surrounding setting, while for general d, a relative GROW bound with a new CSC-type result is established, including asymptotic growth-rate expressions GROW_n = n underline{D} − (d−1)/2 log n + O(1). The work highlights the role of boundary geometry, KL-behavior, and information-theoretic constructs (e.g., Shtarkov, NML) in designing powerful, anytime-valid tests and provides a roadmap for future extensions to higher dimensions and refined asymptotics. Overall, the paper contributes a unified information-theoretic framework linking e-values, CSC bounds, and asymptotic growth, with potential impact on sequential testing and model-selection contexts that require robust, highly efficient testing under optional stopping.

Abstract

We consider growth-optimal e-variables with maximal e-power, both in an absolute and relative sense, for simple null hypotheses for a -dimensional random vector, and multivariate composite alternatives represented as a set of -dimensional means . These include, among others, the set of all distributions with mean in , and the exponential family generated by the null restricted to means in . We show how these optimal e-variables are related to Csiszár-Sanov-Chernoff bounds, first for the case that is convex (these results are not new; we merely reformulate them) and then for the case that `surrounds' the null hypothesis (these results are new).

Paper Structure

This paper contains 20 sections, 11 theorems, 67 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Background on GROW e-variables
  3. Simple $\mathcal{H}_0$ and the Pythagorean Property
  4. The Rôle of Exponential Families
  5. Condition ALT-$\text{\tt M}_1$:
  6. Condition ALT-$\mathcal{H}_1$:
  7. Convex $\text{\tt M}_1$
  8. The Connection between Exponential Families and the Pythagorean Theorem
  9. CSC (Chernoff-Sanov-Csiszár) for convex $\text{\tt M}_1$
  10. Surrounding $\text{\tt M}_1$
  11. GROW for $d=1$
  12. Alternative Optimality Criteria: minimax redundancy and regret
  13. CSC Theorem for surrounding $\mathcal{H}_1$
  14. Analyzing the CSC Result --- Different Partitions $\{\text{\tt M}_{1,r}: r \in \mathcal{R}\}$
  15. Asymptotic expression of growth rate and regret
  16. ...and 5 more sections

Key Result

Theorem 1

GrunwaldHK19 Suppose that (a) $D(P_1 \| P_0)< \infty$ for all $P_1 \in\mathcal{H}_1, P_0 \in \mathcal{H}_0$ and (b) is achieved by some $P_1^*, P_0$, then we have Here $p_1^*$ is the density of $P_1^*$, which exists by the finite KL assumption.

Figures (3)

  • Figure 1: Convex $\mathtt{M}_1$. $\mathtt{M}_1$ is the mean parameter set that $\mathcal{H}_1$ is compatible with, and $\mathtt{M}$ is the mean parameter space of the exponential family generated from $P_0$. The setting of this figure satisfies Condition ALT-$\text{\tt M}_1$.
  • Figure 2: Surrounding, nice, $\mathtt{M}_1$ with a finite nice partition into convex sets. This figure is obtained by taking $P_0$ a Gamma distribution on $X$ and defining $Y= (Y_1,Y_2) = (\log X,X-c)$ for a constant $c > 1$. Then $\mathcal{E}$ is a translated Gamma family with sufficient statistic $Y$ and mean-value space $\text{\tt M}= \{(y_1,y_2): y_1 \in \mathbb{R}, y_2 = e^{y_1} - c \}$ (unlike in Figure \ref{['fig:enter-label']}, we have $\text{\tt M}_1 \subset \text{\tt M}$ here).
  • Figure 3: Surrounding, nice $\mathtt{M}_1$ that cannot be partitioned into a finite number of convex sets (again we show the translated Gamma family).

Theorems & Definitions (24)

  • Definition 1
  • Theorem 1
  • Proposition 1
  • proof
  • Example 1
  • Proposition 2
  • proof
  • Example 2
  • Theorem 2
  • proof
  • ...and 14 more