Minimum Volume Conformal Sets for Multivariate Regression

TL;DR

This paper tackles the problem of constructing valid multivariate predictive sets with finite-sample guarantees while minimizing their volume. It introduces Minimum-Volume Conformal Sets (MVCS), an optimization-driven framework that learns the smallest-volume region defined by arbitrary norm balls, including learning the norm parameter p and multi-norm configurations, and jointly optimizes the predictor and the uncertainty representation. The approach combines DC-based and convex-relaxation techniques to handle nonconvex volume minimization, and extends to local adaptivity and regression by using residual-based centers and covariate-dependent transformations, with conformalization to preserve coverage. Empirical results on synthetic and real datasets show MVCS yields tighter, well-calibrated prediction sets and competitive computational efficiency compared with baselines, highlighting its practical impact for reliable multivariate uncertainty quantification.

Abstract

Conformal prediction provides a principled framework for constructing predictive sets with finite-sample validity. While much of the focus has been on univariate response variables, existing multivariate methods either impose rigid geometric assumptions or rely on flexible but computationally expensive approaches that do not explicitly optimize prediction set volume. We propose an optimization-driven framework based on a novel loss function that directly learns minimum-volume covering sets while ensuring valid coverage. This formulation naturally induces a new nonconformity score for conformal prediction, which adapts to the residual distribution and covariates. Our approach optimizes over prediction sets defined by arbitrary norm balls, including single and multi-norm formulations. Additionally, by jointly optimizing both the predictive model and predictive uncertainty, we obtain prediction sets that are tight, informative, and computationally efficient, as demonstrated in our experiments on real-world datasets.

Figures (9)

• Figure 1: MVCSs obtained using the first-order optimization strategy described in Section \ref{['subsec:p:norms']} with coverage level $0.95$. The left figure is a single-norm MVCS fit with $p = 1.98$ for Gaussian data, the middle one is a single-norm MVCS fit with $p = 0.94$ for exponential data, while the right one is a multi-norm MVCS optimized over four different norms, yielding region-specific values of $p = 0.50, 0.49, 2.52, 1.11$, for a combination of a uniform distribution over three unit balls with norms $0.5$, $3$ and $1$.
• Figure 2: Comparison of the MVCSs obtained via the convex relaxation problem \ref{['eq:min:vol:convex']} and the DCA method \ref{['eq:DCA']}. The solid lines represent the boundaries of $\mathbb{B}(p, M, \mu)$, with dots corresponding to i.i.d. samples from the underlying distribution. (Left) Gaussian distribution: $Y\sim \mathcal{N}\left(0, \left({10.90.91}\right) \right)$ with coverage level $0.90$ and $p = 2$. Both methods produce similar sets, aligning well with the empirical covariance structure. (Right) Exponential distribution: $Y\sim \mathcal{E}(1)^{\otimes 2}$ with coverage level $0.80$ and $p = 10$. While the convex relaxation results in a more elongated set influenced by extreme values, the DCA solution better captures the dense central region of the distribution.
• Figure 3: (Left) Comparison of MVCS sets obtained via the first-order optimization strategy described in Section \ref{['subsec:p:norms']} for solving problem \ref{['eq:min:vol:exact:p']} and the DCA approach with $p=2$. Both methods achieve the target coverage level of $0.95$, but the learned MVCS optimizes $p = 1.11$, leading to a more adaptive shape. (Right) MVCS obtained via \ref{['eq:min:vol:exact:p']} for a dataset exhibiting strong anisotropy. The optimization procedure selects $p = 0.58$, resulting in an elongated set that better captures the structure of the data while achieving the target coverage level of $0.90$.
• Figure 4: Visualization of MVCSs obtained via problem \ref{['eq:min:vol:exact:p']} in three dimensions. (Left) Prediction set learned for a uniform distribution over the 0.5-radius ball, with optimized $p=0.56$ adapting to the anisotropic structure of the data. (Right) MVCS learned for an exponential distribution, where $p=8.19$ results in an axis-aligned structure that reflects the concentration of the data.
• Figure 5: (Left) Comparison of MVCS sets obtained using a single learned norm versus a multi-norm formulation. The single-norm MVCS is learned with $p = 1.21$, while the multi-norm MVCS optimizes over four different norms, yielding region-specific values of $p = 0.68, 0.56, 2.52, 2.59$. (Right) A separate example of multi-norm MVCS with two learned norms, $p = 0.96, 2.90$. In both cases, the multi-norm approach leads to a more adaptive shape that captures the anisotropic structure of the data while maintaining the prescribed coverage level of $0.90$.

Theorems & Definitions (13)

• Proposition 2.1: Reformulation of the MVCS problem
• proof
• Remark 2.2: MVCS for the DCA and convex relaxation problem
• Remark 2.3: Connection to pinball loss
• Corollary 2.4: Single-norm MVCS
• Remark 2.5: Unconstrained single-norm MVCS
• Proposition 2.6: Multi-norm MVCS
• proof
• Remark 2.7: Unconstrained multi-norm MVCS
• Proposition 3.1: Locally adaptive MVCS