Optimal Anytime-Valid Tests for Composite Nulls

TL;DR

The paper tackles the design of level-$\alpha$ power-one tests for composite nulls in an i.i.d. setting by establishing and achieving a fundamental information-theoretic lower bound based on $\gamma^*(P,\mathcal{P}_0)$. It develops two constructive schemes: (i) a finite-alphabet test using a universal $\mathrm{e}$-process built from a Dirichlet (Jeffreys) mixture and running null MLE to achieve asymptotic optimality, and (ii) a general-alphabet DV-based $\mathrm{e}$-process that optimizes over a rich function class $\mathcal{F}$ via empirical saddle-point approximations, with a nonparametric Hölder density example using RKHS that attains the same first-order optimality. The paper also discusses practical computational strategies, including stochastic saddle-point methods and GAN-like heuristics for large-scale problems, and outlines how the results extend to compact convex nulls and nonparametric settings. Together, these results provide a principled, scalable framework for anytime-valid hypothesis testing across parametric and nonparametric regimes, achieving the theoretical lower bound on expected stopping time in the $\alpha\to0$ limit. The work advances the design of optimal, sequential tests that are valid at any stopping time, with clear guidance for implementation and potential extensions to non-i.i.d. data and high-dimensional settings.

Abstract

We consider the problem of designing optimal level-$α$ power-one tests for composite nulls. Given a parameter $α\in (0,1)$ and a stream of $\mathcal{X}$-valued observations $\{X_n: n \geq 1\} \overset{i.i.d.}{\sim} P$, the goal is to design a level-$α$ power-one test $τ_α$ for the null $H_0: P \in \mathcal{P}_0 \subset \mathcal{P}(\mathcal{X})$. Prior works have shown that any such $τ_α$ must satisfy $\mathbb{E}_P[τ_α] \geq \tfrac{\log(1/α)}{γ^*(P, \mathcal{P}_0)}$, where $γ^*(P, \mathcal{P}_0)$ is the so-called $\mathrm{KL}_{\inf}$ or minimum divergence of $P$ to the null class. In this paper, our objective is to develop and analyze constructive schemes that match this lower bound as $α\downarrow 0$. We first consider the finite-alphabet case~($|\mathcal{X}| = m < \infty$), and show that a test based on \emph{universal} $e$-process~(formed by the ratio of a universal predictor and the running null MLE) is optimal in the above sense. The proof relies on a Donsker-Varadhan~(DV) based saddle-point representation of $\mathrm{KL}_{\inf}$, and an application of Sion's minimax theorem. This characterization motivates a general method for arbitrary $\mathcal{X}$: construct an $e$-process based on the empirical solutions to the saddle-point representation over a sufficiently rich class of test functions. We give sufficient conditions for the optimality of this test for compact convex nulls, and verify them for Hölder smooth density models. We end the paper with a discussion on the computational aspects of implementing our proposed tests in some practical settings.

Figures (1)

• Figure 1: The diagram above illustrates the data processing inequality for relative entropy polyanskiy2025information, which says that the distance between two input distributions (as measured by $d_{\text{KL}}$) can never be smaller than the distance between the corresponding output distributions, after passing through a common Markov kernel. In our case, the channel corresponds to the decision rule $\boldsymbol{1}_{\tau_\alpha < \infty}$, which naturally leads to the conclusion recalled in \ref{['fact:lower-bound']}, since it implies that $\mathbb{E}_P[\tau_\alpha] d_{\text{KL}}(P \parallel Q) \geq \log(1/\alpha)$ for any level-$\alpha$ power-one test $\tau_\alpha$, and for distributions $Q \in \mathcal{P}_0$ and $P \not \in \mathcal{P}_0$.

Theorems & Definitions (28)

• Definition 1: First-Order Optimality
• Remark 1
• Theorem 2.1
• Definition 2: DV $e$-process
• Remark 2
• Theorem 3.1
• Remark 3
• Remark 4
• Remark 5
• Remark 6