Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Testing Imprecise Hypotheses

Lucas Kania, Tudor Manole, Larry Wasserman, Sivaraman Balakrishnan

TL;DR

This work develops a rigorous minimax framework for tolerant testing of imprecise hypotheses, where the null distribution is allowed a neighborhood of radius $ε_0$ and alternatives lie at distance at least $ε_1$. Focusing on Gaussian sequence, smooth Gaussian white-noise, and density models, it derives sharp upper and lower bounds on the critical separation and reveals regime structures—free tolerance, interpolation, and functional estimation—for various norms, notably $ℓ_1$ and smooth $ℓ_p$ norms. It shows the classical χ^2 statistic is suboptimal in tolerant settings and proposes practical tests, including plug-in and debiased plug-in statistics, that achieve minimax rates. The results connect to estimation theory via moment-matching dualities and extend to systematic-uncertainty contexts in high-energy physics, offering a robust framework for robust goodness-of-fit testing in scientific applications.

Abstract

Many scientific applications involve testing theories that are only partially specified. This task often amounts to testing the goodness-of-fit of a candidate distribution while allowing for reasonable deviations from it. The tolerant testing framework provides a systematic way of constructing such tests. Rather than testing the simple null hypothesis that data was drawn from a candidate distribution, a tolerant test assesses whether the data is consistent with any distribution that lies within a given neighborhood of the candidate. As this neighborhood grows, the tolerance to misspecification increases, while the power of the test decreases. In this work, we characterize the information-theoretic trade-off between the size of the neighborhood and the power of the test, in several canonical models. On the one hand, we characterize the optimal trade-off for tolerant testing in the Gaussian sequence model, under deviations measured in both smooth and non-smooth norms. On the other hand, we study nonparametric analogues of this problem in smooth regression and density models. Along the way, we establish the sub-optimality of the classical chi-squared statistic for tolerant testing, and study simple alternative hypothesis tests.

Testing Imprecise Hypotheses

TL;DR

This work develops a rigorous minimax framework for tolerant testing of imprecise hypotheses, where the null distribution is allowed a neighborhood of radius and alternatives lie at distance at least . Focusing on Gaussian sequence, smooth Gaussian white-noise, and density models, it derives sharp upper and lower bounds on the critical separation and reveals regime structures—free tolerance, interpolation, and functional estimation—for various norms, notably and smooth norms. It shows the classical χ^2 statistic is suboptimal in tolerant settings and proposes practical tests, including plug-in and debiased plug-in statistics, that achieve minimax rates. The results connect to estimation theory via moment-matching dualities and extend to systematic-uncertainty contexts in high-energy physics, offering a robust framework for robust goodness-of-fit testing in scientific applications.

Abstract

Many scientific applications involve testing theories that are only partially specified. This task often amounts to testing the goodness-of-fit of a candidate distribution while allowing for reasonable deviations from it. The tolerant testing framework provides a systematic way of constructing such tests. Rather than testing the simple null hypothesis that data was drawn from a candidate distribution, a tolerant test assesses whether the data is consistent with any distribution that lies within a given neighborhood of the candidate. As this neighborhood grows, the tolerance to misspecification increases, while the power of the test decreases. In this work, we characterize the information-theoretic trade-off between the size of the neighborhood and the power of the test, in several canonical models. On the one hand, we characterize the optimal trade-off for tolerant testing in the Gaussian sequence model, under deviations measured in both smooth and non-smooth norms. On the other hand, we study nonparametric analogues of this problem in smooth regression and density models. Along the way, we establish the sub-optimality of the classical chi-squared statistic for tolerant testing, and study simple alternative hypothesis tests.
Paper Structure (42 sections, 69 theorems, 429 equations)

This paper contains 42 sections, 69 theorems, 429 equations.

Table of Contents

  1. Introduction
  2. Overview of main results
  3. Related Work
  4. The minimax framework for tolerant testing
  5. The Gaussian sequence model
  6. Tolerant Testing under the l1 norm
  7. Reduction to functional estimation under the l1 norm
  8. Interpolation regime under the l1 norm
  9. Lower-bounds via moment-matching distributions
  10. Lower bound for the free tolerance regime
  11. Lower bound for the functional estimation regime
  12. Lower bound for the interpolation regime
  13. Testing under general smooth lp norms
  14. Relationship to functional estimation
  15. Lower bound on the free tolerance regime
  16. ...and 27 more sections

Key Result

Theorem 1

Given $\epsilon_0 \geq 0$, let $\lambda=\sqrt{n}\epsilon_0$. Then, the hypotheses eq:gsn_seq_expo can be tested with nontrivial power if and only if where we hide polylogarithmic factors in $d$.

Theorems & Definitions (101)

  • Theorem 1: Informal version of thm:gaussian_testing_l1 in sec:simple_null_rates
  • Theorem 2: Informal version of lemma:upper_bound_lp_even in sec:smooth_lp_norm
  • Lemma 1: Suboptimality of the chi-squared test
  • Proposition 1: ingsterTestingHypothesisWhich2001, Optimal tolerant testing for small deviations
  • Theorem 3
  • Lemma 2: Testing by learning
  • Lemma 3
  • Lemma 4
  • Lemma 5: wuMinimaxRatesEntropy2016
  • Theorem 4
  • ...and 91 more