Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Finite-Sample Strong Converse for Binary Hypothesis Testing via (Reverse) Rényi Divergence

Roberto Bruno, Adrien Vandenbroucque, Amedeo Roberto Esposito

Abstract

This work investigates binary hypothesis testing between $H_0\sim P_0$ and $H_1\sim P_1$ in the finite-sample regime under asymmetric error constraints. By employing the ``reverse" Rényi divergence, we derive novel non-asymptotic bounds on the Type II error probability which naturally establish a strong converse result. Furthermore, when the Type I error is constrained to decay exponentially with a rate $c$, we show that the Type II error converges to 1 exponentially fast if $c$ exceeds the Kullback-Leibler divergence $D(P_1\|P_0)$, and vanishes exponentially fast if $c$ is smaller. Finally, we present numerical examples demonstrating that the proposed converse bounds strictly improve upon existing finite-sample results in the literature.

A Finite-Sample Strong Converse for Binary Hypothesis Testing via (Reverse) Rényi Divergence

Abstract

This work investigates binary hypothesis testing between and in the finite-sample regime under asymmetric error constraints. By employing the ``reverse" Rényi divergence, we derive novel non-asymptotic bounds on the Type II error probability which naturally establish a strong converse result. Furthermore, when the Type I error is constrained to decay exponentially with a rate , we show that the Type II error converges to 1 exponentially fast if exceeds the Kullback-Leibler divergence , and vanishes exponentially fast if is smaller. Finally, we present numerical examples demonstrating that the proposed converse bounds strictly improve upon existing finite-sample results in the literature.
Paper Structure (14 sections, 6 theorems, 32 equations, 5 figures)

This paper contains 14 sections, 6 theorems, 32 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Contributions
  3. Preliminaries and notation
  4. Rényi Divergence
  5. Error characterisation via Rényi
  6. Converse Bounds
  7. Achievability Bound
  8. Relationship between Type I and II error probabilities
  9. Comparisons
  10. Proof of Theorem \ref{['theorem:beta_lower_bound_renyi_additive']}
  11. Proof of Corollary \ref{['cor:1']}
  12. Proof of Theorem \ref{['theorem:beta_upper_bound_renyi']}
  13. Proof of Theorem \ref{['th:phase_transition']}
  14. Additional Comparisons

Key Result

Theorem 1

Let $P_1^{n}$ and $P_0^{n}$ be mutually absolutely continuous. For any $\varepsilon \in(0,1)$ it holds that

Figures (5)

  • Figure 1: Bernoulli testing ($P_0=\text{Bern}(1/2)$ vs $P_1=\text{Bern}(1/2+\delta)$, with $\delta=0.01$) under two Type I error ($\varepsilon$) regimes: constant (top) and exponential (bottom). The $x$-axis denotes the sample size $n$, and the $y$-axis the Type II error.
  • Figure 2: Gaussian testing ($P_0=\mathcal{N}(2,1)$ vs $P_1=\mathcal{N}(2+\delta,1)$, with $\delta=0.05$) under two Type I error ($\varepsilon$) regimes: constant (top) and exponential (bottom). The $x$-axis denotes the sample size $n$, and the $y$-axis the Type II error.
  • Figure 3: (Top row) Bernoulli testing ($P_0=\text{Bern}(1/2)$ vs $P_1=\text{Bern}(1/2+\delta)$, with $\delta=0.01$) under three Type I error ($\varepsilon$) regimes: constant (left), linear (middle), and exponential (right). (Bottom row) Gaussian testing ($P_0=\mathcal{N}(2,1)$ vs $P_0=\mathcal{N}(2+\delta,1)$, with $\delta=0.05$) under three Type I error ($\varepsilon$) regimes: constant (left), linear (middle), and exponential (right). The $x$-axis denotes the sample size $n$, and the $y$-axis the Type II error.
  • Figure 4: (Top row) Bernoulli testing ($P_0=\text{Bern}(1/2)$ vs $P_1=\text{Bern}(1/2+\delta)$, with $\delta=0.1$) under three Type I error ($\varepsilon$) regimes: constant (left), linear (middle), and exponential (right). (Bottom row) Gaussian testing ($P_0=\mathcal{N}(2,1)$ vs $P_0=\mathcal{N}(2+\delta,1)$, with $\delta=0.1$) under three Type I error ($\varepsilon$) regimes: constant (left), linear (middle), and exponential (right). The $x$-axis denotes the sample size $n$, and the $y$-axis the Type II error.
  • Figure 5: (Top row) Bernoulli testing ($P_0=\text{Bern}(1/2)$ vs $P_1=\text{Bern}(1/2+\delta)$, with $\delta=0.2$) under three Type I error ($\varepsilon$) regimes: constant (left), linear (middle), and exponential (right). (Bottom row) Gaussian testing ($P_0=\mathcal{N}(2,1)$ vs $P_0=\mathcal{N}(2+\delta,1)$, with $\delta=0.3$) under three Type I error ($\varepsilon$) regimes: constant (left), linear (middle), and exponential (right). The $x$-axis denotes the sample size $n$, and the $y$-axis the Type II error.

Theorems & Definitions (7)

  • Definition 1
  • Theorem 1
  • Corollary 1
  • Theorem 2
  • Theorem 3
  • Lemma 1
  • Lemma 2