Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Single Trajectory Conformal Prediction

Brian Lee, Nikolai Matni

TL;DR

This analysis characterizes the graceful degradation in RCPS performance as data becomes nearly arbitrarily dependent and nonstationary, subject only to a mild requirement that the underlying process is causal.

Abstract

We study the performance of risk-controlling prediction sets (RCPS), an empirical risk minimization-based formulation of conformal prediction, with a single trajectory of temporally correlated data from an unknown stochastic dynamical system. First, we use the blocking technique to show that RCPS attains performance guarantees similar to those enjoyed in the iid setting whenever data is generated by asymptotically stationary and contractive dynamics. Next, we use the decoupling technique to characterize the graceful degradation in RCPS guarantees when the data generating process deviates from stationarity and contractivity. We conclude by discussing how these tools could be used toward a unified analysis of online and offline conformal prediction algorithms, which are currently treated with very different tools.

Single Trajectory Conformal Prediction

TL;DR

This analysis characterizes the graceful degradation in RCPS performance as data becomes nearly arbitrarily dependent and nonstationary, subject only to a mild requirement that the underlying process is causal.

Abstract

We study the performance of risk-controlling prediction sets (RCPS), an empirical risk minimization-based formulation of conformal prediction, with a single trajectory of temporally correlated data from an unknown stochastic dynamical system. First, we use the blocking technique to show that RCPS attains performance guarantees similar to those enjoyed in the iid setting whenever data is generated by asymptotically stationary and contractive dynamics. Next, we use the decoupling technique to characterize the graceful degradation in RCPS guarantees when the data generating process deviates from stationarity and contractivity. We conclude by discussing how these tools could be used toward a unified analysis of online and offline conformal prediction algorithms, which are currently treated with very different tools.
Paper Structure (13 sections, 7 theorems, 40 equations)

This paper contains 13 sections, 7 theorems, 40 equations.

Table of Contents

  1. Introduction
  2. Related Work
  3. Conformal prediction with dependent data.
  4. Statistical learning with dependent data.
  5. Problem Formulation
  6. Notation
  7. Blocked Risk-Controlling Prediction Sets
  8. Preliminaries
  9. Results
  10. Decoupled Risk-Controlling Prediction Sets
  11. Preliminaries
  12. Results
  13. Discussion

Key Result

Proposition 1

The $\upbeta$-mixing coefficient of $X_{1:\infty}$ is upper bounded by which tends to $0$ as $k \to \infty$. Hence strictly stable LTI systems are $\upbeta$-mixing to $\Pi=N(0,\Sigma_\infty)$.

Theorems & Definitions (14)

  • Definition 1: $\beta$- and $\upbeta$-mixing processes yu_rates_1994kuznetsov_generalization_2017
  • Example 1: Strictly stable LTI tu_least-squares_2018
  • Proposition 1: Strictly stable LTI systems are $\upbeta$-mixing tu_least-squares_2018
  • Proposition 2: Blocking technique yu_rates_1994kuznetsov_generalization_2017tu_least-squares_2018
  • Theorem 1: Blocked RCPS, $\Pi$.
  • proof : Proof of Theorem 1
  • Example 2: Strictly stable LTI tu_least-squares_2018
  • Corollary 1: Blocked RCPS, $\mathbf{P}_{T+k}$
  • proof : Proof of Corollary 1
  • Definition 2: Adapted process pena_decoupling_1999
  • ...and 4 more