Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Validity and efficiency of the conformal CUSUM procedure

Vladimir Vovk, Ilia Nouretdinov, Alex Gammerman

TL;DR

This work studies the validity and efficiency of a conformal CUSUM approach for sequential change detection under minimal assumptions. It constructs asymptotically optimal conformal test martingales (CTMs) via the canonical CAO betting function and proves that the CAO CTM yields perfect validity, matching the standard likelihood-ratio martingale in one-step distributions. The authors derive closed-form CAO betting functions for Gaussian changes, investigate efficiency through experiments (Bernoulli and Gaussian shifts), and provide a preliminary theoretical bound in the Bernoulli case linking the CTM to the LR approach. The findings show that conformal CUSUM offers robust validity with competitive detection performance, while conformal e-testing can exhibit broken validity in some scenarios, motivating a focus on CTM-based methods and future theoretical refinements.

Abstract

In this paper we study the validity and efficiency of a conformal version of the CUSUM procedure for change detection both experimentally and theoretically.

Validity and efficiency of the conformal CUSUM procedure

TL;DR

This work studies the validity and efficiency of a conformal CUSUM approach for sequential change detection under minimal assumptions. It constructs asymptotically optimal conformal test martingales (CTMs) via the canonical CAO betting function and proves that the CAO CTM yields perfect validity, matching the standard likelihood-ratio martingale in one-step distributions. The authors derive closed-form CAO betting functions for Gaussian changes, investigate efficiency through experiments (Bernoulli and Gaussian shifts), and provide a preliminary theoretical bound in the Bernoulli case linking the CTM to the LR approach. The findings show that conformal CUSUM offers robust validity with competitive detection performance, while conformal e-testing can exhibit broken validity in some scenarios, motivating a focus on CTM-based methods and future theoretical refinements.

Abstract

In this paper we study the validity and efficiency of a conformal version of the CUSUM procedure for change detection both experimentally and theoretically.

Paper Structure

This paper contains 9 sections, 4 theorems, 56 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Conformal and standard CUSUM procedures
  3. Asymptotically optimal CTM
  4. Computing the CAO betting function in popular special cases
  5. Efficiency and validity: Experimental results
  6. Efficiency: Theoretical analysis
  7. Conclusion
  8. Proof of Theorem \ref{['thm:LR-vs-CT']}
  9. Martingale multiplicative Chernoff bound

Key Result

Proposition 1

For any pair $(Q_0,Q_1)$ of pre-/post-change distributions, the distribution of the likelihood ratio martingale under $Q_0^{\infty}$ coincides with the distribution of any asymptotically optimal CTM under randomness (not necessarily under $Q_0^{\infty}$).

Figures (7)

  • Figure 1: Left panel: the likelihood ratio martingale, conformal e-/pseudomartingale, and conformal test martingale, as described in text, in the Bernoulli case (with the parameter equal to $0.5$ pre-change and $0.6$ post-change). Right panel: the corresponding CUSUM statistics.
  • Figure 2: Left panel: the three processes as in Fig. \ref{['fig:Bernoulli']}, but with a change from $N(0,1)$ to $N(0.2,1)$. Right panel: the corresponding CUSUM statistics.
  • Figure 3: Left panel: the three processes as in Fig. \ref{['fig:Bernoulli']}, but with a change from $N(0,1)$ to $N(0,1.1)$. Right panel: the corresponding CUSUM statistics.
  • Figure 4: Left panel: the three processes as in Fig. \ref{['fig:Bernoulli']}, but with a change from $N(0,1)$ to $N(0,0.9)$. Right panel: the corresponding CUSUM statistics.
  • Figure 5: Left panel: the validity for the Bernoulli case. Right panel: the validity for the Gaussian case with a change in mean. As described in text, with 10,000 simulations.
  • ...and 2 more figures

Theorems & Definitions (9)

  • Proposition 1
  • proof
  • Proposition 2
  • proof : Proof of Proposition \ref{['prop:opt']}
  • Remark 3
  • Remark 4
  • Theorem 5
  • Theorem 6
  • proof : Proof of Theorem \ref{['thm:MMM']}