Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conformalized Interval Arithmetic with Symmetric Calibration

Rui Luo, Zhixin Zhou

TL;DR

The paper tackles uncertainty quantification for the sum of unknown labels over index sets, a scenario where standard conformal prediction for a single target is insufficient. It introduces Conformal Interval Arithmetic (CIA) and a symmetric calibration scheme to obtain valid prediction sets for sums under group exchangeability, with extensions for overlapping subsets, stratification, and Conformal Quantile Regression to boost efficiency. The approach is demonstrated on two applications—group average prediction and path cost prediction—across multiple real-world datasets, showing reliable coverage and competitive interval lengths compared to baselines. The work provides theoretical guarantees, practical algorithms, and open-source code, enabling robust aggregate uncertainty estimation in networked and grouped data contexts.

Abstract

Uncertainty quantification is essential in decision-making, especially when joint distributions of random variables are involved. While conformal prediction provides distribution-free prediction sets with valid coverage guarantees, it traditionally focuses on single predictions. This paper introduces novel conformal prediction methods for estimating the sum or average of unknown labels over specific index sets. We develop conformal prediction intervals for single target to the prediction interval for sum of multiple targets. Under permutation invariant assumptions, we prove the validity of our proposed method. We also apply our algorithms on class average estimation and path cost prediction tasks, and we show that our method outperforms existing conformalized approaches as well as non-conformal approaches.

Conformalized Interval Arithmetic with Symmetric Calibration

TL;DR

The paper tackles uncertainty quantification for the sum of unknown labels over index sets, a scenario where standard conformal prediction for a single target is insufficient. It introduces Conformal Interval Arithmetic (CIA) and a symmetric calibration scheme to obtain valid prediction sets for sums under group exchangeability, with extensions for overlapping subsets, stratification, and Conformal Quantile Regression to boost efficiency. The approach is demonstrated on two applications—group average prediction and path cost prediction—across multiple real-world datasets, showing reliable coverage and competitive interval lengths compared to baselines. The work provides theoretical guarantees, practical algorithms, and open-source code, enabling robust aggregate uncertainty estimation in networked and grouped data contexts.

Abstract

Uncertainty quantification is essential in decision-making, especially when joint distributions of random variables are involved. While conformal prediction provides distribution-free prediction sets with valid coverage guarantees, it traditionally focuses on single predictions. This paper introduces novel conformal prediction methods for estimating the sum or average of unknown labels over specific index sets. We develop conformal prediction intervals for single target to the prediction interval for sum of multiple targets. Under permutation invariant assumptions, we prove the validity of our proposed method. We also apply our algorithms on class average estimation and path cost prediction tasks, and we show that our method outperforms existing conformalized approaches as well as non-conformal approaches.
Paper Structure (37 sections, 3 theorems, 24 equations, 5 figures, 3 tables, 2 algorithms)

This paper contains 37 sections, 3 theorems, 24 equations, 5 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminary and Problem Setup
  3. Split Conformal Prediction
  4. Problem Setup and a Naive Approach
  5. Conformal Interval Arithmetic
  6. Symmetric Calibration
  7. Extensions
  8. Overlapping Subsets
  9. Stratified Conformal Prediction
  10. Conformal Quantile Regression
  11. Related Work
  12. Multiple Test.
  13. Half Sampling.
  14. Fair Conformal Prediction.
  15. Robust Route Planning.
  16. ...and 22 more sections

Key Result

Proposition 1

Suppose that $Z_k$ is group exchangeable in the sense of for any permutation $\pi$ on $[K+1]$: then the prediction set $\mathcal{C}(S_{K+1})$ in eq:pred:simple satisfies

Figures (5)

  • Figure 1: Diagram illustrating the partition of data into calibration and test sets for Proposition \ref{['prop:simple']} and Theorem \ref{['thm:disjoint']}. The sums within each of $S_1,\dots, S_K, S_{K+1}$ (Top) serve as the calibration data to predict the sum within $S_{K+1}$, mirroring the calibration row for the theorem's diagram (Bottom). As $S^{\textrm{cal}}_{K+1}$ and $S^{\textrm{test}}_{K+1}$ are exchangeable, their sums have the same distribution. Therefore, the prediction set \ref{['eq:pred:symmetric:test']} can be derived similarly to \ref{['eq:pred:symmetric']}.
  • Figure 2: Results for constructing prediction sets for subsets without overlaps (Section \ref{['sec:group']}). CIA achieves guaranteed $1-\alpha$ coverage as well as optimal efficiency on various datasets.
  • Figure 3: Results for constructing prediction sets for subsets with overlaps (Section \ref{['sec:path']}). CIA achieves close to $1-\alpha$ coverage and optimal efficiency on two road traffic datasets.
  • Figure 4: The figure illustrates the positive correlation between the subsets overlap ratio $\delta$ and the coverage gap (the difference between the coverage probability and the desired $1-\alpha$ coverage).
  • Figure 5: Complete results for prediction sets. All CIA variants achieve $1-\alpha$ coverage, with CIA (CQR) Stratified showing optimal efficiency.

Theorems & Definitions (4)

  • Proposition 1
  • Theorem 1
  • Remark
  • Theorem 2