Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bounding Neyman-Pearson Region with $f$-Divergences

Andrew Mullhaupt, Cheng Peng

TL;DR

This work establishes a general, information-theoretic lower bound on the Neyman-Pearson region boundary using $f$-divergences, showing that hockey-stick divergences yield tight, boundary-defining lines. It also derives a closed-form, tensorizable upper bound for the NP boundary via the Chernoff $\alpha$-coefficient and provides constructive methods to realize any given NP boundary with distribution pairs. The results connect the NP region to Bayes error rate and ROC curves, enabling direct transfer of classical results and practical estimation from $f$-divergences. Together, these findings deepen the understanding of the information contained in divergences for statistical inference and classification, with implications for sample complexity, bounds on BER, and distributional robustness.

Abstract

The Neyman-Pearson region of a simple binary hypothesis testing is the set of points whose coordinates represent the false positive rate and false negative rate of some test. The lower boundary of this region is given by the Neyman-Pearson lemma, and is up to a coordinate change, equivalent to the optimal ROC curve. We establish a novel lower bound for the boundary in terms of any $f$-divergence. Since the bound generated by hockey-stick $f$-divergences characterizes the Neyman-Pearson boundary, this bound is best possible. In the case of KL divergence, this bound improves Pinsker's inequality. Furthermore, we obtain a closed-form refined upper bound for the Neyman-Pearson boundary in terms of the Chernoff $α$-coefficient. Finally, we present methods for constructing pairs of distributions that can approximately or exactly realize any given Neyman-Pearson boundary.

Bounding Neyman-Pearson Region with $f$-Divergences

TL;DR

This work establishes a general, information-theoretic lower bound on the Neyman-Pearson region boundary using -divergences, showing that hockey-stick divergences yield tight, boundary-defining lines. It also derives a closed-form, tensorizable upper bound for the NP boundary via the Chernoff -coefficient and provides constructive methods to realize any given NP boundary with distribution pairs. The results connect the NP region to Bayes error rate and ROC curves, enabling direct transfer of classical results and practical estimation from -divergences. Together, these findings deepen the understanding of the information contained in divergences for statistical inference and classification, with implications for sample complexity, bounds on BER, and distributional robustness.

Abstract

The Neyman-Pearson region of a simple binary hypothesis testing is the set of points whose coordinates represent the false positive rate and false negative rate of some test. The lower boundary of this region is given by the Neyman-Pearson lemma, and is up to a coordinate change, equivalent to the optimal ROC curve. We establish a novel lower bound for the boundary in terms of any -divergence. Since the bound generated by hockey-stick -divergences characterizes the Neyman-Pearson boundary, this bound is best possible. In the case of KL divergence, this bound improves Pinsker's inequality. Furthermore, we obtain a closed-form refined upper bound for the Neyman-Pearson boundary in terms of the Chernoff -coefficient. Finally, we present methods for constructing pairs of distributions that can approximately or exactly realize any given Neyman-Pearson boundary.
Paper Structure (41 sections, 13 theorems, 87 equations, 5 figures)

This paper contains 41 sections, 13 theorems, 87 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Tensorization.
  4. Binary hypothesis testing.
  5. Randomized test.
  6. Neyman-Pearson region.
  7. Lower Bound for Neyman-Pearson Region
  8. Upper Bound for Neyman-Pearson Boundary
  9. Realization and Approximate Realization of the Neyman-Pearson Region
  10. Connection to Bayes Error Rate
  11. Conjugate function.
  12. Connection to ROC Curve
  13. Equivalence to best possible ROC curve.
  14. Visualization
  15. Shape of lower bounds.
  16. ...and 26 more sections

Key Result

Lemma 1

An extreme point of the Neyman-Pearson region corresponds to a nonrandomized test. Let $E$ be the set that characterizes the nonrandomized test. We have that

Figures (5)

  • Figure 1: Lower bounds for Neyman-Pearson region generated by various $f$-divergences. For each $f$-divergence, $3$ different values of the divergence are used for illustration. The divergence values are labeled along the curve. The black line is the line of ignorance, which is attained by the family of randomized tests that randomly accepts the hypothesis $H_0$ with probability $\alpha$.
  • Figure 2: Refined closed-form upper bound for Neyman-Pearson boundary generated by Hellinger distance. $\rho_\frac{1}{2} = 0.8$.
  • Figure 3: Supporting lines to Neyman-Pearson region corresponding to hockey-stick divergences with different $\gamma$.
  • Figure 4: Lower bounds for Neyman-Pearson region corresponding to squared Hellinger distance with $n$ samples. $\rho = 0.99$.
  • Figure 5: Lower bounds for Neyman-Pearson region for various divergences between concentric univariate Gaussians.

Theorems & Definitions (29)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Lemma 1
  • Theorem 1: Convex lower bound for Neyman-Pearson region in terms of $f$-divergences.
  • Corollary 1: Lower bound for Neyman-Pearson region by reversed $f$-divergences.
  • Example 1: Hockey-stick divergence.
  • Proposition 1: Supporting lines characterizing Neyman-Pearson region.
  • Example 2: Total variation distance.
  • ...and 19 more