Structured Prediction with Abstention via the Lovász Hinge

TL;DR

This work analyzes the Lovász hinge as a surrogate for structured prediction and demonstrates its general inconsistency with the intended target unless the evaluation function is modular. By embedding the Lovász hinge into the structured abstain problem, the authors identify a consistent surrogate target and construct a family of calibrated links that work uniformly over all polymatroids. They prove a tight embedding result after excluding abstain-on-one-coordinate predictions, and provide experiments showing abstention regions align with uncertainty to aid interpretability. The multiclass extension via BEP encoding yields a dimension-efficient, consistent surrogate for a natural multiclass structured abstain objective, offering principled handling of abstention in structured settings.

Abstract

The Lovász hinge is a convex loss function proposed for binary structured classification, in which k related binary predictions jointly evaluated by a submodular function. Despite its prevalence in image segmentation and related tasks, the consistency of the Lovász hinge has remained open. We show that the Lovász hinge is inconsistent with its desired target unless the set function used for evaluation is modular. Leveraging the embedding framework of Finocchiaro et al. (2024), we find the target loss for which the Lovász hinge is consistent. This target, which we call the structured abstain problem, is a variant of selective classification for structured prediction that allows one to abstain on any subset of the k binary predictions. We derive a family of link functions, each of which is simultaneously consistent for all polymatroids, a subset of submodular set functions. We then give sufficient conditions on the polymatroid for the structured abstain problem to be tightly embedded by the Lovász hinge, meaning no target prediction is redundant. We experimentally demonstrate the potential of the structured abstain problem for interpretability in structured classification tasks. Finally, for the multiclass setting, we show that one can combine the binary encoding construction of Ramaswamy et al. (2018) with our link construction to achieve an efficient consistent surrogate for a natural multiclass generalization of the structured abstain problem.

Figures (7)

• Figure 1: (Left) A plot of all $P_{\pi ,y}$ regions and (Right) $V_{\pi}$ labels in the positive orthant where $k=2$.
• Figure 2: $\hat{\Psi}(u)$ for $u \in \mathbb{R}_+^3$ and $\epsilon = \frac{1}{6}$. Each colored region connected to a particular node corresponds to a $v \in \{0,1 \}^3 \subseteq \mathcal{V}$ and at a point $u$, a calibrated link must link to one of the $v$ in the region.
• Figure 3: The link envelope $\hat{\Psi}$ (top left) constructed with respect to $\|\cdot\|_\infty$ and link functions $\psi^\tau$ where $\tau =0$ (top middle), $\tau =1$ (top right), $\tau =.45$ (bottom left), $\tau =.5$ (bottom middle), $\tau =.55$ (bottom right) for $k=2$ and $\epsilon=\frac{1}{4}$. The envelope $\hat{\Psi}$ is pictured for $u \in \mathbb{R}_+^2$, with each region labeled by the value of $\hat{\Psi}$; a link is calibrated if it always links to one of the nodes in the region. The values for the link functions $\psi^\tau$ are given by the unique point $v \in \mathcal{V}$ that each depicted region contains. In particular, the link $\psi^\tau$ for all $\tau \in [0,1]$ satisfy the constraints from $\hat{\Psi}$ (top left) and thus are calibrated.
• Figure : $\tau = 1$
• Figure : $\tau = 1$

Theorems & Definitions (48)

• Definition 1: Elicitation
• Definition 2: Calibration
• Definition 3: Indirect elicitation
• Lemma 4
• Definition 5
• Definition 6: Embedding
• Theorem 7: finocchiaro2022embedding
• Lemma 7
• Lemma 7
• Lemma 7