Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Condition Conformal Selection

Qingyang Hao, Wenbo Liao, Bingyi Jing, Hongxin Wei

TL;DR

This work extends conformal selection to multi-condition settings, enabling FDR-controlled candidate selection under conjunctive and disjunctive constraints by introducing regionally monotone nonconformity scores for intervals and a global BH-based testing scheme. The proposed MCCS framework guarantees finite-sample FDR control across diverse target structures, including multi-interval and multivariate targets, and is validated through extensive simulations and real-data experiments spanning NLP, computer vision, multimodal tasks, and multi-class problems. Key contributions include a conjunctive-specific nonconformity construction, a global BH procedure for disjunctive targets, theoretical FDR guarantees, and extensions to more complex combinations and multivariate responses. The approach demonstrates robust performance, preserving FDR control while achieving competitive power across modalities and task types, with practical implications for drug discovery, precision medicine, and LLM alignment where resource constraints and multiple selection criteria are common.

Abstract

Selecting high-quality candidates from large-scale datasets is critically important in resource-constrained applications such as drug discovery, precision medicine, and the alignment of large language models. While conformal selection methods offer a rigorous solution with False Discovery Rate (FDR) control, their applicability is confined to single-threshold scenarios (i.e., y > c) and overlooks practical needs for multi-condition selection, such as conjunctive or disjunctive conditions. In this work, we propose the Multi-Condition Conformal Selection (MCCS) algorithm, which extends conformal selection to scenarios with multiple conditions. In particular, we introduce a novel nonconformity score with regional monotonicity for conjunctive conditions and a global Benjamini-Hochberg (BH) procedure for disjunctive conditions, thereby establishing finite-sample FDR control with theoretical guarantees. The integration of these components enables the proposed method to achieve rigorous FDR-controlled selection in various multi-condition environments. Extensive experiments validate the superiority of MCCS over baselines, its generalizability across diverse condition combinations, different real-world modalities, and multi-task scalability.

Multi-Condition Conformal Selection

TL;DR

This work extends conformal selection to multi-condition settings, enabling FDR-controlled candidate selection under conjunctive and disjunctive constraints by introducing regionally monotone nonconformity scores for intervals and a global BH-based testing scheme. The proposed MCCS framework guarantees finite-sample FDR control across diverse target structures, including multi-interval and multivariate targets, and is validated through extensive simulations and real-data experiments spanning NLP, computer vision, multimodal tasks, and multi-class problems. Key contributions include a conjunctive-specific nonconformity construction, a global BH procedure for disjunctive targets, theoretical FDR guarantees, and extensions to more complex combinations and multivariate responses. The approach demonstrates robust performance, preserving FDR control while achieving competitive power across modalities and task types, with practical implications for drug discovery, precision medicine, and LLM alignment where resource constraints and multiple selection criteria are common.

Abstract

Selecting high-quality candidates from large-scale datasets is critically important in resource-constrained applications such as drug discovery, precision medicine, and the alignment of large language models. While conformal selection methods offer a rigorous solution with False Discovery Rate (FDR) control, their applicability is confined to single-threshold scenarios (i.e., y > c) and overlooks practical needs for multi-condition selection, such as conjunctive or disjunctive conditions. In this work, we propose the Multi-Condition Conformal Selection (MCCS) algorithm, which extends conformal selection to scenarios with multiple conditions. In particular, we introduce a novel nonconformity score with regional monotonicity for conjunctive conditions and a global Benjamini-Hochberg (BH) procedure for disjunctive conditions, thereby establishing finite-sample FDR control with theoretical guarantees. The integration of these components enables the proposed method to achieve rigorous FDR-controlled selection in various multi-condition environments. Extensive experiments validate the superiority of MCCS over baselines, its generalizability across diverse condition combinations, different real-world modalities, and multi-task scalability.

Paper Structure

This paper contains 37 sections, 4 theorems, 27 equations, 14 figures, 10 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Multi-Condition Conformal Selection
  4. Conditions and Selection Target
  5. conformal p-value
  6. Error Accumulation in Two-Step cfBH Methods
  7. Proposed Method
  8. Addressing Conjunctive Conditions
  9. Addressing Disjunctive Conditions
  10. Addressing More Combination Conditions
  11. Extension to Multivariate Response Settings
  12. Experiments
  13. Compare with Baselines
  14. Experiments on More Combination Conditions
  15. Real Data Applications
  16. ...and 22 more sections

Key Result

Corollary 3.1

Applying two separate cfBH procedures Ying2023cfbh for boundary points and combining results via intersection (for the target interval $(c_1, c_2)$) or union (for $(-\infty, c_1) \cup (c_2, \infty)$) does not guarantee control of the false discovery rate at a prespecified level q, where $c_1 < c_2$

Figures (14)

  • Figure 1: Illustration of Single- and Multi-Condition.
  • Figure 2: Results across varying $q$ for Tasks 1-6.
  • Figure 3: Visualization of nlp.
  • Figure 4: Results on nlp and cv Tasks.
  • Figure 5: Results on vqa Tasks.
  • ...and 9 more figures

Theorems & Definitions (8)

  • Definition 3.1: Regional Monotonicity
  • Corollary 3.1
  • Remark 3.1: Explanation for Corollary \ref{['co-failure']}
  • Proposition 4.1
  • Remark 4.1
  • Corollary 4.1
  • Theorem 4.1
  • proof : Proof of Theorem \ref{['theo1']}