Efficient Quantification of Time-Series Prediction Error: Optimal Selection Conformal Prediction

TL;DR

This work tackles risk quantification for time-series predictions in multi-step control by advancing Conformal Prediction (CP) through Optimal Selection Conformal Prediction (OSCP). OSCP learns a parameterized, time-step-wise nonconformity score via a mixed-integer linear program to minimize the average CP-region radius while preserving validity, enabling norm-ball shaped regions that are convex and efficiently computed. The authors prove validity and efficiency guarantees, demonstrate significant reductions in CP-region size (up to around 16–17%) and dramatic runtime speedups (up to thousands of times faster) across synthetic and real-world datasets, and show that ellipsoid-norms offer further gains in non-Gaussian or high-dimensional settings. The method is practical for safe learning-based MPC and multi-stage safety verification, providing a scalable, distribution-free uncertainty quantification tool for trajectory-level predictions.

Abstract

Uncertainty is almost ubiquitous in safety-critical autonomous systems due to dynamic environments and the integration of learning-based components. Quantifying this uncertainty--particularly for time-series predictions in multi-stage optimization--is essential for safe control and verification tasks. Conformal Prediction (CP) is a distribution-free uncertainty quantification tool with rigorous finite-sample guarantees, but its performance relies on the design of the nonconformity measure, which remains challenging for time-series data. Existing methods either overfit on small datasets, or are computationally intensive on long-time-horizon problems and/or large datasets. To overcome these issues, we propose a new parameterization of the score functions and formulate an optimization program to compute the associated parameters. The optimal parameters directly lead to norm-ball regions that constitute minimal-average-radius conformal sets. We then provide a reformulation of the underlying optimization program to enable faster computation. We provide theoretical proofs on both the validity and efficiency of predictors constructed based on the proposed approach. Numerical results on various case studies demonstrate that our method outperforms state-of-the-art methods in terms of efficiency, with much lower computational requirements.

Figures (5)

• Figure 1: 2D-regression example
• Figure 2: time-series data with $d=2, T=2$
• Figure 4: Performance visualization on the Particle Dataset ($\sigma=0.05$). The dashed reference line in the Coverage graph denotes the target confidences, and only methods with coverage curves at or above this line achieve the target coverages. In the Volume graph, curves closer to the bottom indicate better performances (less conservative).
• Figure 5: Case studies: Particle Datasets. The dashed reference line denotes the target confidences, and only methods with coverage curves at or above this line achieve the target coverages. The shaded region of each curve is the $\pm$ 1 standard error region. In Area graphs, lower curves indicate better performance.
• Figure 6: Case studies: Drone Dataset & Covid-19 Dataset. The dashed reference line denotes the target confidences, and only methods with coverage curves at or above this line achieve the target coverages. The shaded region of each curve is the $\pm$ 1 standard error region. In Length/Volume graphs, lower curves indicate better performance.