Bridging conformal prediction and scenario optimization

TL;DR

This paper builds a formal bridge between conformal prediction (CP) and the scenario approach (SA) by showing how vanilla CP can be aligned with SA to bound the expected probability of constraint violation in scenario programs, and conversely how ranking nonconformity scores can be interpreted as a one-dimensional SA with discarded constraints to recover vanilla CP guarantees. It proves that with an appropriate nonconformity choice, CP set predictors can reproduce SA bounds on violation probabilities, and demonstrates that SA results on fully-supported, discarding-enabled programs yield vanilla CP and calibration-conditional CP guarantees. The work also derives practical relationships, such as quantile-based interpretations of CP thresholds and explicit sample-size bounds, enabling transfer of results and techniques between CP and SA. It points to future directions, including handling non-fully-supported problems via regularization and developing adaptive CP methods guided by SA insights, to broaden the applicability of these theoretical connections.

Abstract

Conformal prediction and scenario optimization constitute two important classes of statistical learning frameworks to certify decisions made using data. They have found numerous applications in control theory, machine learning and robotics. Despite intense research in both areas, and apparently similar results, a clear connection between these two frameworks has not been established. By focusing on the so-called vanilla conformal prediction, we show rigorously how to choose appropriate score functions and set predictor map to recover well-known bounds on the probability of constraint violation associated with scenario programs. We also show how to treat ranking of nonconformity scores as a one-dimensional scenario program with discarded constraints, and use such connection to recover vanilla conformal prediction guarantees on the validity of the set predictor. We also capitalize on the main developments of the scenario approach, and show how we could analyze calibration conditional conformal prediction under this lens. Our results establish a theoretical bridge between conformal prediction and scenario optimization.

Figures (2)

• Figure 1: Connections between conformal prediction and scenario optimization. From conformal prediction to scenario approach: By appropriate choice of nonconformity scores and a set predictor in vanilla conformal prediction, we recover well-known results on the expected value of the probability of constraint violation in scenario programs (Theorem \ref{['thm:CP-SP-connection-average-violation']}). From scenario approach to conformal prediction: By interpreting the ranking of nonconformity scores as an one-dimensional scenario program with discarded constraints, we derive vanilla conformal prediction results (Theorem \ref{['thm:scen_conf_van']}). We also view the calibration conditional conformal prediction under the scenario approach lens, using the results on the cumulative distribution of the probability of constraint violation in scenario programs (Theorem \ref{['thm:scen_conf_cond']}).
• Figure 2: Scenario program pictorial construction: Given a collection of ordered samples (nonconformity scores), $R_1,\ldots,R_m$, discard the $r$ highest ones (the ones that fall in the red region). The maximum out of the remaining ones is the solution of an one-dimensional scenario program, denoted by $R_p=C(R_1,\ldots,R_m,r)$.

Theorems & Definitions (18)

• Definition 1
• Theorem 1: Proposition 1.2 Balasubramanian2014
• Corollary 1
• Definition 2: Fully-supported scenario programs
• Theorem 2: Theorem 4, Romao2023
• Theorem 3
• proof
• Proposition 1
• proof
• Theorem 4