Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Second-Order Asymptotics of Two-Sample Tests

K V Harsha, Jithin Ravi, Tobias Koch

TL;DR

This work establishes a unified divergence-based framework for two-sample testing, revealing that the optimal first-order error exponent is $2 D_B(P_1,P_2)$ irrespective of the chosen divergence. For invariant divergences, it proves a second-order equivalence with the Gutman test (and hence with the GLRT), deriving a generalized chi-square limit for the divergence statistic and explicit expressions for the second-order term $\beta''$ using the KL-variance at the minimizer $P^{*}$. The Gutman test is shown to be the GLRT for robust GoF testing, connecting two-sample problems to uncertainty-class GoF in a principled way. For non-invariant divergences, the first-order optimality persists, but second-order behavior can differ and remains generally open, highlighting a fundamental trade-off between invariance and finer finite-sample performance. Overall, the paper provides a rigorous asymptotic characterization that links divergence-based two-sample testing, robust GoF, and GLRT theory, with practical implications for designing universal tests with predictable error behavior.

Abstract

In two-sampling testing, one observes two independent sequences of independent and identically distributed random variables distributed according to the distributions $P_1$ and $P_2$ and wishes to decide whether $P_1=P_2$ (null hypothesis) or $P_1\neq P_2$ (alternative hypothesis). The Gutman test for this problem compares the empirical distributions of the observed sequences and decides on the null hypothesis if the Jensen-Shannon (JS) divergence between these empirical distributions is below a given threshold. This paper proposes a generalization of the Gutman test, termed \emph{divergence test}, which replaces the JS divergence by an arbitrary divergence. For this test, the exponential decay of the type-II error probability for a fixed type-I error probability is studied. First, it is shown that the divergence test achieves the optimal first-order exponent, irrespective of the choice of divergence. Second, it is demonstrated that the divergence test with an invariant divergence achieves the same second-order asymptotics as the Gutman test. In addition, it is shown that the Gutman test is the GLRT for the two-sample testing problem, and a connection between two-sample testing and robust goodness-of-fit testing is established.

Second-Order Asymptotics of Two-Sample Tests

TL;DR

This work establishes a unified divergence-based framework for two-sample testing, revealing that the optimal first-order error exponent is irrespective of the chosen divergence. For invariant divergences, it proves a second-order equivalence with the Gutman test (and hence with the GLRT), deriving a generalized chi-square limit for the divergence statistic and explicit expressions for the second-order term using the KL-variance at the minimizer . The Gutman test is shown to be the GLRT for robust GoF testing, connecting two-sample problems to uncertainty-class GoF in a principled way. For non-invariant divergences, the first-order optimality persists, but second-order behavior can differ and remains generally open, highlighting a fundamental trade-off between invariance and finer finite-sample performance. Overall, the paper provides a rigorous asymptotic characterization that links divergence-based two-sample testing, robust GoF, and GLRT theory, with practical implications for designing universal tests with predictable error behavior.

Abstract

In two-sampling testing, one observes two independent sequences of independent and identically distributed random variables distributed according to the distributions and and wishes to decide whether (null hypothesis) or (alternative hypothesis). The Gutman test for this problem compares the empirical distributions of the observed sequences and decides on the null hypothesis if the Jensen-Shannon (JS) divergence between these empirical distributions is below a given threshold. This paper proposes a generalization of the Gutman test, termed \emph{divergence test}, which replaces the JS divergence by an arbitrary divergence. For this test, the exponential decay of the type-II error probability for a fixed type-I error probability is studied. First, it is shown that the divergence test achieves the optimal first-order exponent, irrespective of the choice of divergence. Second, it is demonstrated that the divergence test with an invariant divergence achieves the same second-order asymptotics as the Gutman test. In addition, it is shown that the Gutman test is the GLRT for the two-sample testing problem, and a connection between two-sample testing and robust goodness-of-fit testing is established.
Paper Structure (24 sections, 9 theorems, 70 equations)

This paper contains 24 sections, 9 theorems, 70 equations.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Notation
  4. Divergence-Based Two-Sample Tests
  5. Divergence
  6. Divergence Test
  7. Robust Goodness-of-Fit Testing and the GLRT
  8. Asymptotics of Divergence Tests with Invariant Divergences
  9. Proof Outline of Theorem \ref{['Thm:unknownPdivtest']}
  10. Asymptotics of Divergence Tests with Non-Invariant Divergences
  11. Proof of Proposition \ref{['Thm:glrt_rob']}
  12. Proof of Lemma \ref{['cov_lemma']}
  13. Proof of Theorem \ref{['Thm:unknownPdivtest']}
  14. Upper Bound on $\beta_n$
  15. Lower Bound on $\beta_n$
  16. ...and 9 more sections

Key Result

Proposition 1

Consider the two-sample testing problem eqs:2sample_GoF with $\mathcal{C}$ defined in eq:def_class. The GLRT of this problem decides on $H_0$ if $D_{\textnormal{JS}}(T_{X^{n}}\| T_{Y^{n}})$ is below a given threshold $r>0$, and it decides on $H_1$ otherwise. Similarly, the test statistic eq:ROB_test

Theorems & Definitions (14)

  • Definition 1
  • Remark 1
  • Definition 2
  • Proposition 1
  • proof
  • Theorem 1
  • Theorem 2
  • Definition 3
  • Lemma 1
  • Proposition 2
  • ...and 4 more