Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A complete characterization of testable hypotheses

Martin Larsson, Johannes Ruf, Aaditya Ramdas

TL;DR

This work resolves when nontrivial hypothesis tests exist without a dominating measure by proving a necessary and sufficient condition based on weak-* closures in the ba space of bounded finitely additive measures. It shows that a test with strictly higher worst-case power than level exists if and only if the total-variation distance between the weak-* closures of the convex hulls of $\mathcal{P}$ and $\mathcal{Q}$ exceeds the desired gap, with the minimax risk given by $R(\mathcal{P},\mathcal{Q}) = 1 - \text{d}_{\text{TV}}(\overline{\text{co}^*}(\mathcal{P}), \overline{\text{co}^*}(\mathcal{Q}))$, and that the closures must be taken in ba to capture finitely additive phenomena. The paper also analyzes the singleton-$\mathcal{Q}$ case, clarifies the relationship to the effective null hypothesis, and discusses generalized tests and practical implications for robust testing. By employing Fan's minimax theorem and carefully characterizing closures in ba, the authors unify dominated and non-dominated testing regimes and illuminate the fundamental role of finitely additive measures in complete testability theory. These results extend Le Cam's program and provide a precise, constructive criterion for testability in broad nonparametric settings, with implications for uniformly powered e-variables and robust procedure design.

Abstract

We revisit a fundamental question in hypothesis testing: given two sets of probability measures $\mathcal{P}$ and $\mathcal{Q}$, when does a nontrivial (i.e.\ strictly unbiased) test for $\mathcal{P}$ against $\mathcal{Q}$ exist? Le~Cam showed that, when $\mathcal{P}$ and $\mathcal{Q}$ have a common dominating measure, a test that has power exceeding its level by more than $\varepsilon$ exists if and only if the convex hulls of $\mathcal{P}$ and $\mathcal{Q}$ are separated in total variation distance by more than $\varepsilon$. The requirement of a dominating measure is frequently violated in nonparametric statistics. In a passing remark, Le~Cam described an approach to address more general scenarios, but he stopped short of stating a formal theorem. This work completes Le~Cam's program, by presenting a matching necessary and sufficient condition for testability: for the aforementioned theorem to hold without assumptions, one must take the closures of the convex hulls of $\mathcal{P}$ and $\mathcal{Q}$ in the space of bounded finitely additive measures. We provide simple elucidating examples, and elaborate on various subtle measure theoretic and topological points regarding compactness and achievability.

A complete characterization of testable hypotheses

TL;DR

This work resolves when nontrivial hypothesis tests exist without a dominating measure by proving a necessary and sufficient condition based on weak-* closures in the ba space of bounded finitely additive measures. It shows that a test with strictly higher worst-case power than level exists if and only if the total-variation distance between the weak-* closures of the convex hulls of and exceeds the desired gap, with the minimax risk given by , and that the closures must be taken in ba to capture finitely additive phenomena. The paper also analyzes the singleton- case, clarifies the relationship to the effective null hypothesis, and discusses generalized tests and practical implications for robust testing. By employing Fan's minimax theorem and carefully characterizing closures in ba, the authors unify dominated and non-dominated testing regimes and illuminate the fundamental role of finitely additive measures in complete testability theory. These results extend Le Cam's program and provide a precise, constructive criterion for testability in broad nonparametric settings, with implications for uniformly powered e-variables and robust procedure design.

Abstract

We revisit a fundamental question in hypothesis testing: given two sets of probability measures and , when does a nontrivial (i.e.\ strictly unbiased) test for against exist? Le~Cam showed that, when and have a common dominating measure, a test that has power exceeding its level by more than exists if and only if the convex hulls of and are separated in total variation distance by more than . The requirement of a dominating measure is frequently violated in nonparametric statistics. In a passing remark, Le~Cam described an approach to address more general scenarios, but he stopped short of stating a formal theorem. This work completes Le~Cam's program, by presenting a matching necessary and sufficient condition for testability: for the aforementioned theorem to hold without assumptions, one must take the closures of the convex hulls of and in the space of bounded finitely additive measures. We provide simple elucidating examples, and elaborate on various subtle measure theoretic and topological points regarding compactness and achievability.
Paper Structure (7 sections, 25 theorems, 69 equations)

This paper contains 7 sections, 25 theorems, 69 equations.

Table of Contents

  1. Introduction
  2. Finitely additive measures and proofs of the main results
  3. The case of a singleton ${\mathcal{Q}}$
  4. Generalized tests
  5. Proof of Theorem \ref{['T_Peff']}
  6. Some measure-theoretic results
  7. On the necessity of considering $\textnormal{ba}$: a further example

Key Result

Theorem 1.1

If ${\mathcal{P}}$ and ${\mathcal{Q}}$ have a common dominating measure $\gamma$ (meaning that $\mu \ll \gamma$ and $\nu \ll \gamma$ for all $\mu\in{\mathcal{P}},\nu\in{\mathcal{Q}}$), then for any $\varepsilon \geq 0$, In fact, we have where the minimax risk is achieved by some $\phi^* \in \Phi$, but the infimum in $d_\textnormal{TV}$ is in general not achieved by any $\mu \in \mathop{\mathrm{\

Theorems & Definitions (57)

  • Theorem 1.1
  • Example 1.2
  • Example 1.3
  • Example 1.4
  • Theorem 1.5
  • Proposition 1.6
  • Theorem 1.7
  • Example 1.8
  • Corollary 1.9
  • proof
  • ...and 47 more