Predictive Inference With Fast Feature Conformal Prediction

TL;DR

Conformal Prediction provides distribution-free uncertainty quantification but standard approaches can be slow when leveraging feature-space information via FCP. FFCP replaces the nonlinear head transformation with a first-order Taylor expansion and a gradient-based non-conformity score, delivering substantial speedups while preserving predictive validity. The work proves coverage guarantees and demonstrates shorter bands than Vanilla CP across regression, segmentation, and classification contexts, with about a 50x speed improvement over FCP. Moreover, FFCP generalizes to other CP variants such as CQR and LCP (as FFCQR and FFLCP) and demonstrates practical impact through real-world datasets and segmentation tasks. Overall, FFCP expands the practical deployment of conformal prediction in deep learning by enabling fast, gradient-informed uncertainty quantification that scales to complex models and tasks.

Abstract

Conformal prediction is widely adopted in uncertainty quantification, due to its post-hoc, distribution-free, and model-agnostic properties. In the realm of modern deep learning, researchers have proposed Feature Conformal Prediction (FCP), which deploys conformal prediction in a feature space, yielding reduced band lengths. However, the practical utility of FCP is limited due to the time-consuming non-linear operations required to transform confidence bands from feature space to output space. In this paper, we introduce Fast Feature Conformal Prediction (FFCP), which features a novel non-conformity score and is convenient for practical applications. FFCP serves as a fast version of FCP, in that it equivalently employs a Taylor expansion to approximate the aforementioned non-linear operations in FCP. Empirical validations showcase that FFCP performs comparably with FCP (both outperforming the vanilla version) while achieving a significant reduction in computational time by approximately 50x. The code is available at https://github.com/ElvisWang1111/FastFeatureCP

Figures (5)

• Figure 1: Comparison among Vanilla CP, FCP, and FFCP. FCP and FFCP are more efficient compared to Vanilla CP since they return different band lengths for different individuals. This is done by calculating a non-conformity score in the feature space. Besides, FFCP approximates FCP using a Taylor expansion, which leads to a different non-conformity score and accelerates the transformation from feature space to output space.
• Figure 2: The results of FFCP and FCP in image segmentation tasks show that brighter regions indicate areas of uncertainty. Both FFCP and FCP highlight uncertain regions around the edges, which is informative. FFCP, however, returns bands with sharper boundaries.
• Figure 3: Scatter plot of FCP Score and FFCP Score at different Layers. The relationship between FCP Score and FFCP Score is positively correlated, which indicates that FFCP Score effectively replaces FCP Score.
• Figure 4: Empirical validation of Theorem \ref{['thm: ffcp efficient']}. We plot the score distributions and their corresponding quantiles ($\alpha=0.1$) of Vanilla CP (left) and FFCP (right). Compared to Vanilla CP, the non-conformity scores of FFCP are closer to their quantiles, leading to a shorter band. Compared to Vanilla, FFCP exhibits a more stable distribution with higher quantiles, leading to better performance for FFCP. FFCP selects layer 2 for display.
• Figure 5: Additional visualization results in segmentation task.

Theorems & Definitions (6)

• Theorem 2
• Remark 3: High-dimensional Response
• Theorem 4: Coverage
• Theorem 5: Band Length
• Theorem 6
• proof : Proof of Theorem \ref{['thm: ffcp efficient']}