Multivariate Conformal Selection

TL;DR

Multivariate Conformal Selection (mCS) extends Conformal Selection to multivariate responses by introducing regional monotonicity and two nonconformity-score families, mCS-dist and mCS-learn. It constructs conformal p-values for multivariate targets and applies the Benjamini-Hochberg procedure to guarantee finite-sample FDR control while aiming to maximize selection power. Across simulated and real-world drug-discovery datasets, mCS consistently outperforms adapted baselines while maintaining FDR control, with the learning-based variant showing particular strength for high-dimensional or irregular target regions. This framework enables principled, model-agnostic, multivariate selection in high-stakes scientific tasks, with practical impact in compound screening and other multi-criteria decision problems.

Abstract

Selecting high-quality candidates from large datasets is critical in applications such as drug discovery, precision medicine, and alignment of large language models (LLMs). While Conformal Selection (CS) provides rigorous uncertainty quantification, it is limited to univariate responses and scalar criteria. To address this issue, we propose Multivariate Conformal Selection (mCS), a generalization of CS designed for multivariate response settings. Our method introduces regional monotonicity and employs multivariate nonconformity scores to construct conformal p-values, enabling finite-sample False Discovery Rate (FDR) control. We present two variants: mCS-dist, using distance-based scores, and mCS-learn, which learns optimal scores via differentiable optimization. Experiments on simulated and real-world datasets demonstrate that mCS significantly improves selection power while maintaining FDR control, establishing it as a robust framework for multivariate selection tasks.

Figures (4)

• Figure 1: Observed FDR and power across varying nominal levels for Task 1 (shifted first orthant $R$) and 2 (spherical $R$).
• Figure 2: Scatter plots of 1,000 i.i.d. samples from our data generating processes. Each point represents a sample, with the $x$- and $y$-axes corresponding to its responses entries $\boldsymbol{y}_1$ and $\boldsymbol{y}_2$, respectively. The red and the yellow shaded areas represents the two target regions for the two selection tasks.
• Figure 3: Histograms of 15 responses in the drug dataset. The vertical red lines denote the corresponding cutoffs (for the first task) for each response.
• Figure 4: Description and drug discovery relevance of the responses.

Theorems & Definitions (12)

• Definition 3.1: Regional Monotonicity
• Proposition 3.2
• Remark 3.3: Clarification on conservativeness
• Remark 3.4: Univariate monotonicity as a special case
• Theorem 3.5
• Theorem 4.1
• Proposition 4.2
• Remark 4.3: Subfamilies of the score class
• Remark 4.4: Incorporating pretrained models
• proof : Proof of Proposition \ref{['prop:conservative']}