Non-Asymptotic Analysis of Efficiency in Conformalized Regression

TL;DR

<3-5 sentence high-level summary> The paper analyzes non-asymptotic efficiency of split conformal regression methods, focusing on conformalized quantile regression (CQR) and conformalized median regression (CMR) trained with SGD. It derives finite-sample bounds on the deviation of the calibrated prediction-set length from the oracle interval, showing a rate of $\mathcal{O}\left( \frac{1}{\sqrt{n}} + \frac{1}{\alpha^{2} n} + \frac{1}{\sqrt{m}} + \exp(-\alpha^{2} m) \right)$ that depends on the training size $n$, calibration size $m$, and miscoverage level $\alpha$. The analysis reveals phase transitions across regimes of $\alpha$ and provides data-allocation guidelines to balance training and calibration for desired accuracy. Experiments on synthetic and real-world data validate the theoretical predictions and show robustness across optimizers, non-linear ground truths, and different model classes.

Abstract

Conformal prediction provides prediction sets with coverage guarantees. The informativeness of conformal prediction depends on its efficiency, typically quantified by the expected size of the prediction set. Prior work on the efficiency of conformalized regression commonly treats the miscoverage level $α$ as a fixed constant. In this work, we establish non-asymptotic bounds on the deviation of the prediction set length from the oracle interval length for conformalized quantile and median regression trained via SGD, under mild assumptions on the data distribution. Our bounds of order $\mathcal{O}(1/\sqrt{n} + 1/(α^2 n) + 1/\sqrt{m} + \exp(-α^2 m))$ capture the joint dependence of efficiency on the proper training set size $n$, the calibration set size $m$, and the miscoverage level $α$. The results identify phase transitions in convergence rates across different regimes of $α$, offering guidance for allocating data to control excess prediction set length. Empirical results are consistent with our theoretical findings.

Figures (18)

• Figure 1: Proof outline of Theorem \ref{['thm:cqr-main']}. Full proof deferred to Section \ref{['sec:prf-cqr']}.
• Figure 2: Upper bound orders in Theorem \ref{['thm:cqr-main']} in different regimes of $\alpha$ when $n = \Theta(m)$. Results in lei2018distributionbars2025volume lie in the right most regime (blue).
• Figure 3: The length deviation of conformalized quantile regression in synthetic data experiments.
• Figure 4: The probability density function of $Y|X=x$ for synthetic dataset.
• Figure 5: Log–log regression of length deviation $\Delta$ versus $1/(n\alpha^2)$ for relatively small $\alpha$.

Theorems & Definitions (44)

• Remark 3.1
• Theorem 3.1: Quantile regression error of SGD-trained models
• Remark 3.2
• Theorem 3.2: Efficiency of CQR-SGD
• Remark 3.3
• Remark 3.4
• Remark 4.1
• Theorem 4.1: Efficiency of CMR
• Theorem 1.1: Quantile regression error of SGD-trained models
• Proposition 1.1