Aggregating Conformal Prediction Sets via α-Allocation

TL;DR

COLA advances conformal prediction by aggregating multiple nonconformity scores through data-driven confidence-level allocation, forming an intersection of calibrated prediction sets to reduce average set size under marginal coverage. It develops four variants (COLA-e, COLA-s, COLA-f, COLA-l) balancing efficiency, finite-sample validity, full conformalization, and individualized allocation, with theoretical guarantees including asymptotic optimality and, for COLA-s/f, finite-sample validity. Empirical results on synthetic and real data demonstrate consistently smaller sets than state-of-the-art baselines while maintaining coverage, and COLA-l delivers further gains by tailoring allocations to test-point covariates. The work offers practical, scalable strategies for multi-score conformal aggregation and opens directions for improving COLA-f efficiency and exploring alternative allocation criteria in decision-making contexts.

Abstract

Conformal prediction offers a distribution-free framework for constructing prediction sets with finite-sample coverage. Yet, efficiently leveraging multiple conformity scores to reduce prediction set size remains a major open challenge. Instead of selecting a single best score, this work introduces a principled aggregation strategy, COnfidence-Level Allocation (COLA), that optimally allocates confidence levels across multiple conformal prediction sets to minimize empirical set size while maintaining provable coverage. Two variants are further developed, COLA-s and COLA-f, which guarantee finite-sample marginal coverage via sample splitting and full conformalization, respectively. In addition, we develop COLA-l, an individualized allocation strategy that promotes local size efficiency while achieving asymptotic conditional coverage. Extensive experiments on synthetic and real-world datasets demonstrate that COLA achieves considerably smaller prediction sets than state-of-the-art baselines while maintaining valid coverage.

Figures (9)

• Figure 1: Performance under Cases 1--3 with varying $n$. Top row: coverage; bottom row: prediction set size ratio relative to COLA-e. The black dotted lines mark the nominal coverage level $1-\alpha=0.9$.
• Figure 2: Performance under Case 3 with varying $K$. Left: coverage; right: prediction set size ratio relative to COLA-e. The black dotted line marks the nominal coverage level $1-\alpha=0.9$.
• Figure 3: Performance of COLA-e and COLA-l across locations. Top row: prediction set (black dotted line, oracle conditionally valid set; gray points, data samples). Middle row: coverage (black dotted line, nominal level $1-\alpha=0.9$). Bottom row: prediction set size (black dotted line, size of the oracle conditionally valid set).
• Figure 4: Prediction intervals for the same randomly drawn test instance under COLA-e and EFCP. "Len" denotes the length of the corresponding interval.
• Figure 5: Coverage and size under Case 1.

Theorems & Definitions (24)

• Definition 1: Asymptotically optimal allocation
• Theorem 1: Validity of $\mathrm{COLA\text{-}e}$
• Theorem 2
• Theorem 3: Finite-sample validity of $\mathrm{COLA\text{-}s}$
• Corollary 1
• Theorem 4: Finite-sample validity of $\mathrm{COLA\text{-}f}$
• Remark 1
• Theorem 5
• Theorem 6
• proof : Proof of Theorem \ref{['thm:valid_cola_e']}