Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An information-theoretic lower bound in time-uniform estimation

John C. Duchi, Saminul Haque

TL;DR

The paper establishes information-theoretic lower bounds for time-uniform parameter estimation by reducing estimation to a sequence of increasingly difficult testing problems, leveraging Le Cam’s method and the Bretagnolle-Huber inequality. A Cantor-type parameter construction paired with a quadratic KL expansion yields a universal lower bound of order $t(n) \gtrsim \sqrt{ \frac{\log \log n}{n} }$ that applies to location models, generalized linear models, and semiparametric settings with nuisance parameters, and remains sharp up to constants. The authors also show how a simple doubling strategy can attain matching time-uniform guarantees, up to constants, and they prove the sharpness of the dependence on the confidence level $\alpha$ via Proposition on log-alpha terms. Collectively, the results clarify fundamental limits for time-uniform inference and inform design principles for confidence sequences in sequential analysis. The framework unifies classical two-point lower bounds with modern time-uniform inference across broad model classes including exponential families and semiparametric models.

Abstract

We present an information-theoretic lower bound for the problem of parameter estimation with time-uniform coverage guarantees. Via a new a reduction to sequential testing, we obtain stronger lower bounds that capture the hardness of the time-uniform setting. In the case of location model estimation, logistic regression, and exponential family models, our $Ω(\sqrt{n^{-1}\log \log n})$ lower bound is sharp to within constant factors in typical settings.

An information-theoretic lower bound in time-uniform estimation

TL;DR

The paper establishes information-theoretic lower bounds for time-uniform parameter estimation by reducing estimation to a sequence of increasingly difficult testing problems, leveraging Le Cam’s method and the Bretagnolle-Huber inequality. A Cantor-type parameter construction paired with a quadratic KL expansion yields a universal lower bound of order that applies to location models, generalized linear models, and semiparametric settings with nuisance parameters, and remains sharp up to constants. The authors also show how a simple doubling strategy can attain matching time-uniform guarantees, up to constants, and they prove the sharpness of the dependence on the confidence level via Proposition on log-alpha terms. Collectively, the results clarify fundamental limits for time-uniform inference and inform design principles for confidence sequences in sequential analysis. The framework unifies classical two-point lower bounds with modern time-uniform inference across broad model classes including exponential families and semiparametric models.

Abstract

We present an information-theoretic lower bound for the problem of parameter estimation with time-uniform coverage guarantees. Via a new a reduction to sequential testing, we obtain stronger lower bounds that capture the hardness of the time-uniform setting. In the case of location model estimation, logistic regression, and exponential family models, our lower bound is sharp to within constant factors in typical settings.
Paper Structure (11 sections, 15 theorems, 61 equations)

This paper contains 11 sections, 15 theorems, 61 equations.

Table of Contents

  1. Introduction
  2. Notation
  3. Reduction from estimation to testing
  4. Reduction from estimation to testing
  5. Hardness of the testing problem
  6. Applying the lower bound
  7. Location models
  8. Generalized linear models
  9. Models with nuisance parameters
  10. Attaining the lower bound
  11. Proof of Proposition \ref{['proposition:log-alpha-dependence']}

Key Result

proposition 1

Suppose $\{P_v\}_{v \in \mathcal{V}} \subset \mathcal{P}$ has parameter separation $\{\delta_k\}_{k \in \N}$ and $\hat{\theta}$ is an $(\alpha, t(\cdot))$-estimator. Then for $n_k = \inf\{n : t(n) < \delta_k/2\}$ there exists an $(\alpha, \{n_k\})$-test.

Theorems & Definitions (25)

  • definition 1: $(\alpha, \{n_k\}_{k \in \N})$-test
  • proposition 1
  • proof
  • theorem 1
  • corollary 1
  • lemma 1
  • proof
  • lemma 2: Le Cam's method
  • lemma 3: Bretagnolle-Huber
  • lemma 4: KL-divergence of conditional distributions
  • ...and 15 more