Enhanced Multi-model Online Conformal Prediction

TL;DR

This work addresses the inefficiency of online conformal prediction (CP) when employing multiple models by introducing GMOCP, a graph-based multi-model framework. At each time step, a bipartite graph selects a subset of effective models, from which a single model is chosen to form the CP prediction set; miscoverage probabilities are per-model and updated via scale-free online gradient descent with a pinball loss. The approach reduces computational cost and improves prediction-set efficiency while preserving the required coverage, as demonstrated on synthetic data and TinyImageNet-C. The results indicate smaller prediction sets and faster runtimes compared with existing MOCP baselines, supporting practical deployment in online uncertainty quantification under distribution shifts.

Abstract

Conformal prediction is a framework for uncertainty quantification that constructs prediction sets for previously unseen data, guaranteeing coverage of the true label with a specified probability. However, the efficiency of these prediction sets, measured by their size, depends on the choice of the underlying learning model. Relying on a single fixed model may lead to suboptimal performance in online environments, as a single model may not consistently perform well across all time steps. To mitigate this, prior work has explored selecting a model from a set of candidates. However, this approach becomes computationally expensive as the number of candidate models increases. Moreover, poorly performing models in the set may also hinder the effectiveness. To tackle this challenge, this work develops a novel multi-model online conformal prediction algorithm that reduces computational complexity and improves prediction efficiency. At each time step, a bipartite graph is generated to identify a subset of effective models, from which a model is selected to construct the prediction set. Experiments demonstrate that our method outperforms existing multi-model conformal prediction techniques in terms of both prediction set size and computational efficiency.