Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sequential Multiple Testing: A Second-Order Asymptotic Analysis

Jingyu Liu, Yanglei Song

Abstract

We study sequential multiple testing with independent data streams, where the goal is to identify an unknown subset of signals while controlling commonly used error metrics, including generalized familywise rates and false discovery and non-discovery rates. For these problems, procedures that are first-order optimal are known, in the sense that the ratio of their expected sample size (ESS) to the minimal achievable ESS converges to one as the error tolerance levels vanish. In this work, we develop a unified theory of second-order asymptotic optimality. We establish general sufficient conditions under which second-order Bayesian optimality implies second-order frequentist optimality for broad classes of sequential testing procedures. As a consequence, several procedures previously known to be first-order optimal are shown to be second-order optimal: for every signal configuration, the difference between their ESS and the minimal achievable ESS remains uniformly bounded as the error tolerance levels tend to zero. In addition, we derive a second-order asymptotic expansion of the minimal achievable ESS, which refines the classical first-order approximation by identifying the second-order correction term arising from a boundary-crossing problem for a multidimensional random walk. We apply this result to several commonly used error metrics.

Sequential Multiple Testing: A Second-Order Asymptotic Analysis

Abstract

We study sequential multiple testing with independent data streams, where the goal is to identify an unknown subset of signals while controlling commonly used error metrics, including generalized familywise rates and false discovery and non-discovery rates. For these problems, procedures that are first-order optimal are known, in the sense that the ratio of their expected sample size (ESS) to the minimal achievable ESS converges to one as the error tolerance levels vanish. In this work, we develop a unified theory of second-order asymptotic optimality. We establish general sufficient conditions under which second-order Bayesian optimality implies second-order frequentist optimality for broad classes of sequential testing procedures. As a consequence, several procedures previously known to be first-order optimal are shown to be second-order optimal: for every signal configuration, the difference between their ESS and the minimal achievable ESS remains uniformly bounded as the error tolerance levels tend to zero. In addition, we derive a second-order asymptotic expansion of the minimal achievable ESS, which refines the classical first-order approximation by identifying the second-order correction term arising from a boundary-crossing problem for a multidimensional random walk. We apply this result to several commonly used error metrics.
Paper Structure (33 sections, 32 theorems, 324 equations, 7 figures)

This paper contains 33 sections, 32 theorems, 324 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Frequentist formulation, Optimality, and Minimum ESS
  3. Multiple Testing Error Rates
  4. Asymptotic Optimality
  5. Minimum ESS Characterization
  6. A Framework for Frequentist Optimality via Bayesian Formulations
  7. Bayesian Formulation and Bayesian Optimality
  8. Second-Order Frequentist Optimality via Bayesian Optimality
  9. Second-Order Characterization of the Minimum ESS under the Frequentist Formulation
  10. Applications under Different Error Metrics
  11. Generalized Misclassification Rate
  12. Generalized Familywise Error Rates
  13. False Discovery and Non-discovery Rates
  14. Discussion of Optimality
  15. Numerical Results under Generalized Misclassification Rate
  16. ...and 18 more sections

Key Result

Theorem 1

Suppose that Assumption assumption: lorden holds. Assume that there exists a cost $c_{\boldsymbol{\theta}} > 0$ for each $\boldsymbol{\theta} \in (0,1)^r$, a loss function $\mathcal{W} : \mathcal{A} \times \mathcal{A} \to [0,\infty)$, and a constant $L > 0$ such that the following conditions hold: Then there exists a constant $M > 0$, depending only on $\mathcal{W}$ and not on $\boldsymbol{\theta

Figures (7)

  • Figure 1: Symmetric case: $K=20, m_0=1$. The x-axis in all panels is $|\log_{10}\alpha|$. In the leftmost plots, "S-I" denotes the Sum-Intersection rule (see \ref{['eq: sum intersection rule']}) and "AP" denotes the asymptotic approximation to the smallest achievable ESS. In the middle plots, "Diff." denotes the difference between the ESS of the S-I rule and the two approximations. In the rightmost plots, "Ratio" denotes the ratio of the ESS of the S-I rule to each of the two approximations.
  • Figure 2: The x-axis in all panels is $|\log_{10}\alpha|$. In the leftmost plots, "S-I" denotes the Sum-Intersection rule and "AP" denotes the asymptotic approximation to the smallest achievable ESS. In the middle plots, "Diff." denotes the difference between the ESS of the S-I rule and the two approximations. In the rightmost plots, "Ratio" denotes the ratio of the ESS of the S-I rule to each of the two approximations.
  • Figure 3: Visualization of the subsets appearing in the proof of the second-order optimality of the Sum-Intersection rule.
  • Figure 4: Visualization of the subsets appearing in the proof of the ESS approximation under the generalized misclassification rate.
  • Figure 5: Visualization of the subsets appearing in the proof of the second-order optimality of the Leap rule.
  • ...and 2 more figures

Theorems & Definitions (91)

  • Definition 1: Asymptotic Optimality
  • Definition 2: Asymptotic approximation
  • Remark 1
  • Remark 2
  • Definition 3: Second-order Bayesian optimality
  • Remark 3
  • Theorem 1
  • Remark 4
  • Definition 4
  • Remark 5
  • ...and 81 more