Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

JAPAN: Joint Adaptive Prediction Areas with Normalising-Flows

Eshant English, Christoph Lippert

TL;DR

JAPAN reframes uncertainty quantification by replacing residual-based conformity with density-based scores from normalising flows to construct CP regions. By thresholding the estimated conditional density p(y|x) via log-density thresholds, JAPAN yields geometry-free, potentially disjoint prediction regions that adapt to complex, multimodal distributions while maintaining finite-sample coverage guarantees. The authors provide theoretical results on optimality under rank preservation and approximate bounds when rank is preserved approximately, and demonstrate superior calibration and tighter regions across multivariate regression and time-series forecasting, with several extensions including unconditional, conditional, posterior, latent, and tau-adaptive variants. The approach scales to high-dimensional, structured data and offers a flexible framework for reliable and efficient predictive uncertainty in diverse domains.

Abstract

Conformal prediction provides a model-agnostic framework for uncertainty quantification with finite-sample validity guarantees, making it an attractive tool for constructing reliable prediction sets. However, existing approaches commonly rely on residual-based conformity scores, which impose geometric constraints and struggle when the underlying distribution is multimodal. In particular, they tend to produce overly conservative prediction areas centred around the mean, often failing to capture the true shape of complex predictive distributions. In this work, we introduce JAPAN (Joint Adaptive Prediction Areas with Normalising-Flows), a conformal prediction framework that uses density-based conformity scores. By leveraging flow-based models, JAPAN estimates the (predictive) density and constructs prediction areas by thresholding on the estimated density scores, enabling compact, potentially disjoint, and context-adaptive regions that retain finite-sample coverage guarantees. We theoretically motivate the efficiency of JAPAN and empirically validate it across multivariate regression and forecasting tasks, demonstrating good calibration and tighter prediction areas compared to existing baselines. We also provide several \emph{extensions} adding flexibility to our proposed framework.

JAPAN: Joint Adaptive Prediction Areas with Normalising-Flows

TL;DR

JAPAN reframes uncertainty quantification by replacing residual-based conformity with density-based scores from normalising flows to construct CP regions. By thresholding the estimated conditional density p(y|x) via log-density thresholds, JAPAN yields geometry-free, potentially disjoint prediction regions that adapt to complex, multimodal distributions while maintaining finite-sample coverage guarantees. The authors provide theoretical results on optimality under rank preservation and approximate bounds when rank is preserved approximately, and demonstrate superior calibration and tighter regions across multivariate regression and time-series forecasting, with several extensions including unconditional, conditional, posterior, latent, and tau-adaptive variants. The approach scales to high-dimensional, structured data and offers a flexible framework for reliable and efficient predictive uncertainty in diverse domains.

Abstract

Conformal prediction provides a model-agnostic framework for uncertainty quantification with finite-sample validity guarantees, making it an attractive tool for constructing reliable prediction sets. However, existing approaches commonly rely on residual-based conformity scores, which impose geometric constraints and struggle when the underlying distribution is multimodal. In particular, they tend to produce overly conservative prediction areas centred around the mean, often failing to capture the true shape of complex predictive distributions. In this work, we introduce JAPAN (Joint Adaptive Prediction Areas with Normalising-Flows), a conformal prediction framework that uses density-based conformity scores. By leveraging flow-based models, JAPAN estimates the (predictive) density and constructs prediction areas by thresholding on the estimated density scores, enabling compact, potentially disjoint, and context-adaptive regions that retain finite-sample coverage guarantees. We theoretically motivate the efficiency of JAPAN and empirically validate it across multivariate regression and forecasting tasks, demonstrating good calibration and tighter prediction areas compared to existing baselines. We also provide several \emph{extensions} adding flexibility to our proposed framework.

Paper Structure

This paper contains 34 sections, 4 theorems, 54 equations, 15 figures, 7 tables, 7 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Inductive conformal prediction
  4. Normalising flows
  5. JAPAN: Joint Adaptive Prediction Areas via Normalising-Flows
  6. General setting: multivariate regression
  7. Special case: multi-step time series modelling
  8. Estimating prediction area
  9. Experiments
  10. Time series modelling:
  11. Toy densities
  12. Multivariate regression
  13. Time series data
  14. Extensions
  15. Unconditional Modelling
  16. ...and 19 more sections

Key Result

Proposition 1

Let $p(y \mid x)$ be the true conditional density, and let $f_\theta(y \mid x)$ be the model-estimated density. If there exists a strictly increasing function $g: \mathbb{R}^+ \to \mathbb{R}^+$ such that $f_\theta(y \mid x) = g(p(y \mid x))$ for all $y$, then the conformal prediction region defined

Figures (15)

  • Figure 1: For the spiral density, only JAPAN produces compact regions that align with high-density areas, as the prediction area closely follows the spiral, whereas other baselines fail to capture this.
  • Figure 2: Visualisation of different densities (Moons, Rings, and Checkerboard) at $\epsilon = 0.1$. Only JAPAN captures the modes and avoids low-density regions. Hard constrained areas, such as Elliptical and Rectangular, naturally fail at capturing the true shape. CONTRA encapsulates low-density regions, completely ignoring discontinuities in the density. PCP while captures discontinuity includes some low-density regions due to the soft constraints.
  • Figure 3: Computed areas per dataset (Checkerboard, Moons, Circles, Spiral) for all the methods computed over 25 random data splits. Each bar's length signifies the standard deviations, with methods sorted by average area for each dataset. JAPAN consistently has the lowest areas and hence is more informative.
  • Figure 4: Prediction areas per dataset (Energy, RF2D, RF4D, SCM) for the top 5 methods computed over 25 random data splits. Each bar's length signifies the standard deviations, with methods sorted by average area for each dataset. JAPAN consistently has the lowest areas and hence is more informative.
  • Figure 5: Prediction areas per dataset (COVID, Particle-1, Drone, Pedestrian) for the top 5 methods computed over 25 random data splits. Each bar's length signifies the standard deviations, with methods sorted by average area for each dataset. JAPAN consistently has the lowest areas and hence is more informative.
  • ...and 10 more figures

Theorems & Definitions (4)

  • Proposition 1: Optimality under Rank Preservation
  • Proposition 2: Approximate Optimality under Approximate Rank Preservation
  • Proposition 3: Area/Volume computation
  • Proposition 4: Bounded Misranking Leads to Bounded Excess Area