Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conformal e-testing

Vladimir Vovk, Ilia Nouretdinov, Alex Gammerman

Abstract

There is a useful counterpart of conformal prediction for e-values, called conformal e-prediction. Conformal prediction can serve as basis for testing the assumption of exchangeability, leading to conformal testing. Similarly, conformal e-prediction can also serve as basis for testing. The resulting conformal e-testing looks very different from but inherits some strengths of conformal testing; it even has some advantages over conformal testing. In this paper we discuss systematically both strengths and limitations of conformal e-testing.

Conformal e-testing

Abstract

There is a useful counterpart of conformal prediction for e-values, called conformal e-prediction. Conformal prediction can serve as basis for testing the assumption of exchangeability, leading to conformal testing. Similarly, conformal e-prediction can also serve as basis for testing. The resulting conformal e-testing looks very different from but inherits some strengths of conformal testing; it even has some advantages over conformal testing. In this paper we discuss systematically both strengths and limitations of conformal e-testing.

Paper Structure

This paper contains 9 sections, 8 theorems, 54 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Conformal e-prediction
  3. Conformal e-testing in the online setting
  4. Conformal e-testing in the batch setting
  5. The conformal CUSUM e-procedure: validity
  6. The conformal CUSUM e-procedure: efficiency
  7. Conclusion
  8. Proofs for Sect. \ref{['sec:CUSUM-validity']}
  9. Proof for Sect. \ref{['sec:CUSUM-efficiency']}

Key Result

Proposition 1

For any conformal e-predictor $f$ and any $n$, if the sequence $Z_1,Z_2,\dots$ is exchangeable, then where $\mathcal{F}$ is the exchangeable filtration and $E_n$ is the conformal e-/variable eq:E (with "$=$" in place of "$\le$" in eq:validity if $f$ is admissible).

Figures (2)

  • Figure 1: Five stochastic processes, as described in text, in the Bernoulli case (with the parameter $0.5$ before the changepoint and $0.6$ after the changepoint). Left panel: the paths of the processes. Right panel: the paths of the corresponding CUSUM statistics.
  • Figure 2: Five processes as in Figure \ref{['fig:Bernoulli']} but for the Cauchy distributions with the location and scale parameters $(0,1)$ before the changepoint and $(0,0.7)$ after the changepoint. Left panel: the raw processes. Right panel: the corresponding CUSUM statistics.

Theorems & Definitions (13)

  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3
  • proof
  • Proposition 4
  • proof
  • Proposition 5
  • Proposition 6
  • ...and 3 more