Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Convex Loss Function for Set Prediction with Optimal Trade-offs Between Size and Conditional Coverage

Francis Bach

TL;DR

This work introduces a convex loss for set-valued predictions derived from the Lovász–Choquet framework, enabling a principled trade-off between the size of predicted uncertainty sets and conditional coverage. By representing sets as level sets of a real-valued function and integrating over a continuum of Lagrange multipliers, the authors obtain a tractable, fully convex objective that yields all size-conditional coverage trade-offs and supports randomized predictions. They derive efficient optimization schemes (SGD and iteratively-reweighted least squares) and demonstrate improvements over marginal-coverage baselines on synthetic classification and regression tasks, while also discussing conformalization for marginal guarantees. The paper also offers a rich set of examples, alternative area-based criteria, and practical guidelines for implementing the approach in large-scale settings. Overall, it provides a unified convex-optimization pathway to quantify and control uncertainty sets in high-dimensional prediction problems with explicit size-coverage considerations.

Abstract

We consider supervised learning problems in which set predictions provide explicit uncertainty estimates. Using Choquet integrals (a.k.a. Lov{á}sz extensions), we propose a convex loss function for nondecreasing subset-valued functions obtained as level sets of a real-valued function. This loss function allows optimal trade-offs between conditional probabilistic coverage and the ''size'' of the set, measured by a non-decreasing submodular function. We also propose several extensions that mimic loss functions and criteria for binary classification with asymmetric losses, and show how to naturally obtain sets with optimized conditional coverage. We derive efficient optimization algorithms, either based on stochastic gradient descent or reweighted least-squares formulations, and illustrate our findings with a series of experiments on synthetic datasets for classification and regression tasks, showing improvements over approaches that aim for marginal coverage.

A Convex Loss Function for Set Prediction with Optimal Trade-offs Between Size and Conditional Coverage

TL;DR

This work introduces a convex loss for set-valued predictions derived from the Lovász–Choquet framework, enabling a principled trade-off between the size of predicted uncertainty sets and conditional coverage. By representing sets as level sets of a real-valued function and integrating over a continuum of Lagrange multipliers, the authors obtain a tractable, fully convex objective that yields all size-conditional coverage trade-offs and supports randomized predictions. They derive efficient optimization schemes (SGD and iteratively-reweighted least squares) and demonstrate improvements over marginal-coverage baselines on synthetic classification and regression tasks, while also discussing conformalization for marginal guarantees. The paper also offers a rich set of examples, alternative area-based criteria, and practical guidelines for implementing the approach in large-scale settings. Overall, it provides a unified convex-optimization pathway to quantify and control uncertainty sets in high-dimensional prediction problems with explicit size-coverage considerations.

Abstract

We consider supervised learning problems in which set predictions provide explicit uncertainty estimates. Using Choquet integrals (a.k.a. Lov{á}sz extensions), we propose a convex loss function for nondecreasing subset-valued functions obtained as level sets of a real-valued function. This loss function allows optimal trade-offs between conditional probabilistic coverage and the ''size'' of the set, measured by a non-decreasing submodular function. We also propose several extensions that mimic loss functions and criteria for binary classification with asymmetric losses, and show how to naturally obtain sets with optimized conditional coverage. We derive efficient optimization algorithms, either based on stochastic gradient descent or reweighted least-squares formulations, and illustrate our findings with a series of experiments on synthetic datasets for classification and regression tasks, showing improvements over approaches that aim for marginal coverage.
Paper Structure (50 sections, 5 theorems, 54 equations, 5 figures)

This paper contains 50 sections, 5 theorems, 54 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Trade-off between conditional coverage and size.
  3. Lagrangian formulation.
  4. Contributions
  5. Related work
  6. Pinball loss.
  7. Set predictions.
  8. Density estimation.
  9. Conditional coverage for set prediction.
  10. Convex loss functions for subset-valued functions
  11. Randomized predictions.
  12. Integrated loss functions for additive measures
  13. Lovász extension / Choquet integral
  14. Convexity and submodularity.
  15. Computing values and subgradients of $v$.
  16. ...and 35 more sections

Key Result

Lemma 1

If $V$ is a finite additive measure, then for any $h:{\mathcal{Y}}\to \mathbb{R}_-$ and additive measure $Q$ on ${\mathcal{Y}}$, we have (with $B^{\sf c}$ denoting the complement of the set $B$):

Figures (5)

  • Figure 1: Effect of concave penalty. Left to right: posterior probabilities, optimal prediction functions for three concave penalties, for ${\mathcal{X}}=\mathbb{R}$ and $k=5$ classes.
  • Figure 2: Examples of erosion, dilation, opening, closure of a set from the structuring element (left panel) in two dimensions. The size function defined in Eq. (\ref{['eq:morphomat']}) leads to sets that are equal to their closure, thus without small holes compared to the size of the structuring element.
  • Figure 3: Examples of estimation of functions. From left to right: density $\pi$, estimation with increasing radius $r$ of structuring elements (and increasingly larger piecewise constant parts).
  • Figure 4: Given a finite set of points $(V(A(\nu,x)),{\mathbb P}(Y \notin A(\nu,x)|X=x))_{\nu \in \mathbb{R}}$ (black crosses), three different interpolants can be defined, each with its own area.
  • Figure 11: Comparing different estimated coverage at level $\alpha = 0.1$ for the one-dimensional regression problem from Figure \ref{['fig:cover_1d']}. From left to right: optimal set, learned set with a modular penalty, learned set with cover penalty, intervall loss.

Theorems & Definitions (5)

  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Proposition 1
  • Proposition 2