Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Calibratable Disambiguation Loss for Multi-Instance Partial-Label Learning

Wei Tang, Yin-Fang Yang, Weijia Zhang, Min-Ling Zhang

TL;DR

This work tackles calibration in dual-inexact supervision settings by introducing Calibratable Disambiguation Loss (CDL) for multi-instance partial-label learning (MIPL). CDL comes in two instantiations and is designed as a plug-and-play loss that improves both classification accuracy and probability calibration; it integrates with existing attention-based MIPL frameworks. The authors provide a theoretical lower bound showing CDL regularizes the training via a data-driven confidence margin, and they validate the approach with extensive experiments on benchmark and real-world datasets, achieving state-of-the-art accuracy and substantially lower ECE. They further analyze the mechanisms behind CDL’s success, including improved feature aggregation and more reliable label disambiguation, and offer practical guidance for selecting CDL variants and attention mechanisms depending on data complexity.

Abstract

Multi-instance partial-label learning (MIPL) is a weakly supervised framework that extends the principles of multi-instance learning (MIL) and partial-label learning (PLL) to address the challenges of inexact supervision in both instance and label spaces. However, existing MIPL approaches often suffer from poor calibration, undermining classifier reliability. In this work, we propose a plug-and-play calibratable disambiguation loss (CDL) that simultaneously improves classification accuracy and calibration performance. The loss has two instantiations: the first one calibrates predictions based on probabilities from the candidate label set, while the second one integrates probabilities from both candidate and non-candidate label sets. The proposed CDL can be seamlessly incorporated into existing MIPL and PLL frameworks. We provide a theoretical analysis that establishes the lower bound and regularization properties of CDL, demonstrating its superiority over conventional disambiguation losses. Experimental results on benchmark and real-world datasets confirm that our CDL significantly enhances both classification and calibration performance.

Calibratable Disambiguation Loss for Multi-Instance Partial-Label Learning

TL;DR

This work tackles calibration in dual-inexact supervision settings by introducing Calibratable Disambiguation Loss (CDL) for multi-instance partial-label learning (MIPL). CDL comes in two instantiations and is designed as a plug-and-play loss that improves both classification accuracy and probability calibration; it integrates with existing attention-based MIPL frameworks. The authors provide a theoretical lower bound showing CDL regularizes the training via a data-driven confidence margin, and they validate the approach with extensive experiments on benchmark and real-world datasets, achieving state-of-the-art accuracy and substantially lower ECE. They further analyze the mechanisms behind CDL’s success, including improved feature aggregation and more reliable label disambiguation, and offer practical guidance for selecting CDL variants and attention mechanisms depending on data complexity.

Abstract

Multi-instance partial-label learning (MIPL) is a weakly supervised framework that extends the principles of multi-instance learning (MIL) and partial-label learning (PLL) to address the challenges of inexact supervision in both instance and label spaces. However, existing MIPL approaches often suffer from poor calibration, undermining classifier reliability. In this work, we propose a plug-and-play calibratable disambiguation loss (CDL) that simultaneously improves classification accuracy and calibration performance. The loss has two instantiations: the first one calibrates predictions based on probabilities from the candidate label set, while the second one integrates probabilities from both candidate and non-candidate label sets. The proposed CDL can be seamlessly incorporated into existing MIPL and PLL frameworks. We provide a theoretical analysis that establishes the lower bound and regularization properties of CDL, demonstrating its superiority over conventional disambiguation losses. Experimental results on benchmark and real-world datasets confirm that our CDL significantly enhances both classification and calibration performance.

Paper Structure

This paper contains 37 sections, 2 theorems, 22 equations, 10 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multi-Instance Learning
  4. Partial-Label Learning
  5. Multi-Instance Partial-Label Learning
  6. Model Calibration
  7. Preliminaries
  8. Notations
  9. Calibration
  10. The Pitfalls of Naive Focal Losses in MIPL
  11. The Proposed Approach
  12. Aggregation Mechanisms in MIPL
  13. Calibratable Disambiguation Loss
  14. Pseudo-Code of Calibratable Disambiguation Loss
  15. Theoretical Evidence
  16. ...and 22 more sections

Key Result

Theorem 1

Let $\mathcal{L}_{\text{CDL}}$ denote the Calibratable Disambiguation Loss and $\mathcal{L}_{\text{MDL}}$ denote the Momentum-based Disambiguation Loss. For any multi-instance bag $\boldsymbol{X}_i$ with the candidate label set $\mathcal{S}_i$, under the condition that $\Phi(\hat{\boldsymbol{p}}_{i} where $\beta_i = \max_{c^\prime \in \mathcal{S}_i} \hat{p}_{i, c^\prime}^{(t)} - \Phi(\hat{\boldsym

Figures (10)

  • Figure 1: Pathology image classification with crowd-sourced candidate label sets is a MIPL scenario tang2023demipl, where true labels are highlighted in red and false positive labels in black. The labels include LYM (lymphocytes), MUS (smooth muscle), STR (cancer-associated stroma), and TUM (colorectal adenocarcinoma epithelium).
  • Figure 2: Reliability diagrams of (a) DeMipltang2023demipl, (b) EliMipltang2024elimipl, (c) MiplMatang2024miplma, (d) DamCc, (e) SamCc, (f) MamCc, (g) DamCn, (h) SamCn, and (i) MamCn on the C-KMeans test set. The diagrams display mean accuracy (ACC) and expected calibration error (ECE) from ten runs, with (d)-(i) representing our methods. The bar color intensity reflects the number of samples assigned to the corresponding confidence intervals.
  • Figure 3: Reliability diagrams of Samtang2024elimipl with FL or IFL on the FMNIST-MIPL dataset with one false positive label ($r=1$).
  • Figure 4: Framework of MIPL approaches within the embedded-space paradigm.
  • Figure 5: Reliability diagrams of DeMipltang2023demipl, EliMipltang2024elimipl, and MiplMatang2024miplma, and our methods on the Birdsong-MIPL datasets with varying numbers of false positive labels ($r \in \{1,2,3\}$). The bar color intensity indicates that more samples are assigned with the corresponding confidence intervals.
  • ...and 5 more figures

Theorems & Definitions (4)

  • Definition 1: Calibratable Disambiguation Loss
  • Theorem 1: Lower Bound and Regularization Properties of CDL
  • Theorem 1: Lower Bound and Regularization Properties of CDL
  • proof