Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

JANET: Joint Adaptive predictioN-region Estimation for Time-series

Eshant English, Eliot Wong-Toi, Matteo Fontana, Stephan Mandt, Padhraic Smyth, Christoph Lippert

TL;DR

JANET addresses the lack of finite-sample joint uncertainty quantification for time-series by extending generalized inductive conformal prediction to multi-step, univariate and multivariate settings. It introduces two history- or horizon-adaptive non-conformity scores and constructs rectangular joint prediction regions with $K$-FWER control, enabling interpretable, real-time assessment of forecast sequences. The method achieves near-nominal coverage with competitive or narrower interval widths and substantially lower computational cost compared to bootstrap-based approaches, across diverse synthetic and real-world datasets. JANET offers a practical, model-agnostic framework for reliable uncertainty quantification in sequential data, with clear avenues for future extensions such as cross-conformal setups and disjoint prediction regions.

Abstract

Conformal prediction provides machine learning models with prediction sets that offer theoretical guarantees, but the underlying assumption of exchangeability limits its applicability to time series data. Furthermore, existing approaches struggle to handle multi-step ahead prediction tasks, where uncertainty estimates across multiple future time points are crucial. We propose JANET (Joint Adaptive predictioN-region Estimation for Time-series), a novel framework for constructing conformal prediction regions that are valid for both univariate and multivariate time series. JANET generalises the inductive conformal framework and efficiently produces joint prediction regions with controlled K-familywise error rates, enabling flexible adaptation to specific application needs. Our empirical evaluation demonstrates JANET's superior performance in multi-step prediction tasks across diverse time series datasets, highlighting its potential for reliable and interpretable uncertainty quantification in sequential data.

JANET: Joint Adaptive predictioN-region Estimation for Time-series

TL;DR

JANET addresses the lack of finite-sample joint uncertainty quantification for time-series by extending generalized inductive conformal prediction to multi-step, univariate and multivariate settings. It introduces two history- or horizon-adaptive non-conformity scores and constructs rectangular joint prediction regions with -FWER control, enabling interpretable, real-time assessment of forecast sequences. The method achieves near-nominal coverage with competitive or narrower interval widths and substantially lower computational cost compared to bootstrap-based approaches, across diverse synthetic and real-world datasets. JANET offers a practical, model-agnostic framework for reliable uncertainty quantification in sequential data, with clear avenues for future extensions such as cross-conformal setups and disjoint prediction regions.

Abstract

Conformal prediction provides machine learning models with prediction sets that offer theoretical guarantees, but the underlying assumption of exchangeability limits its applicability to time series data. Furthermore, existing approaches struggle to handle multi-step ahead prediction tasks, where uncertainty estimates across multiple future time points are crucial. We propose JANET (Joint Adaptive predictioN-region Estimation for Time-series), a novel framework for constructing conformal prediction regions that are valid for both univariate and multivariate time series. JANET generalises the inductive conformal framework and efficiently produces joint prediction regions with controlled K-familywise error rates, enabling flexible adaptation to specific application needs. Our empirical evaluation demonstrates JANET's superior performance in multi-step prediction tasks across diverse time series datasets, highlighting its potential for reliable and interpretable uncertainty quantification in sequential data.
Paper Structure (43 sections, 2 theorems, 27 equations, 10 figures, 13 tables, 1 algorithm)

This paper contains 43 sections, 2 theorems, 27 equations, 10 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background on conformal prediction
  3. Generalised inductive conformal predictors
  4. Multiple time series
  5. Single time series
  6. Permutation scheme
  7. Validity of ICP with permutations
  8. Remark:
  9. JANET: Joint Adaptive predictioN-region Estimation for Time-series
  10. Remark:
  11. Experiments
  12. Single time series experiments
  13. Monte Carlo simulations
  14. U.S. real gross domestic product (GDP) dataset
  15. Multiple independent time series experiments
  16. ...and 28 more sections

Key Result

Theorem 1

Under mild assumptions on ergodicity and small prediction errors (see sec:assumptions2), for any $\epsilon \in (0, 1)$, the approximate conformal p-value is approximately distributed as follows: for any $\epsilon \in (0, 1)$ and the corresponding conformal set has an approximate coverage $1-\epsilon$, i.e,

Figures (10)

  • Figure 1: Illustration of Non-Overlapping Block (NOB) permutations applied to the calibration portion of a single time series $Z$ of length $L = L_{tr} + L_{cal} = 16$. The first 10 points ($z_1, \dots, z_{10}$) are used for training, and the final 6 points ($z_{11}, \dots, z_{16}$) form the calibration segment. Each row in the figure corresponds to a different permuted version of the calibration sequence $Z_{cal}$, where a block size $b = 1$ is used. The top row represents the identity permutation (i.e., no rotation). Each subsequent row is obtained by rotating the calibration sequence one block to the left compared to the row above it. The arrows indicate how elements are rearranged to produce the next new permutation. For example, the second row shows the permutation $\pi_2$ applied to the identity (first row), and so on.
  • Figure 2: Monte Carlo Experiment Coverage Error (pp) vs Interval Width. The $y$-axis shows the difference in coverage from the target $1-\epsilon$ for $\epsilon=0.2$ in percentage points (pp) and the $x$-axis is the geometric mean of the interval width over the time horizon. The red line represents perfect calibration. Better methods can be found near the red line and to the left (well-calibrated, narrower interval width). The left plot is for the case of $K=1$ whilst the right plot shows $K=3$. The Bonferroni and Scheffé-JPR methods are only applicable for $K=1$. Shapes represent calibration methods and colours signify forecast horizon, $H$.
  • Figure 3: The resulting GDP data after preprocessing (log transform and differencing).
  • Figure 4: Monte Carlo Experiment Coverage Error (pp) vs Interval Width. The $y$-axis shows the difference in coverage from the target $1-\epsilon$ for $\epsilon=0.2$ in percentage points and the $x$-axis is the geometric mean of the interval width over the time horizon. The red line represents perfect calibration. Better methods can be found near the red line and to the left (well-calibrated, narrower interval width). The left plot is for the case of $K=1$ while the right plot shows $K=3$. The Bonferroni and Scheffé-JPR methods are only applicable for $K=1$. Shapes represent calibration methods and colors signify forecast horizon, $H$. For $K=1$ (top row) note that the Bonferroni regions are consistently overconservative (overcoverage) while the Scheffé regions are consistently anticonservative (undercoverage). Meanwhile the bootstrap and JANET JPRs are comparable and generally provide close to the desired coverage and are similar in width.
  • Figure 5: Bar plot for the coverages of different methods, $\epsilon = 0.1$. The desired coverage is 90%. Negative values indicate undercoverage and positive values indicate overcoverage.
  • ...and 5 more figures

Theorems & Definitions (4)

  • Definition 1: Randomised $p$-value
  • Theorem 1: Approximate General Validity of Inductive Conformal Inference
  • proof
  • Theorem 2: General Exact Validity