Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Probabilistic Label Spreading: Efficient and Consistent Estimation of Soft Labels with Epistemic Uncertainty on Graphs

Jonathan Klees, Tobias Riedlinger, Peter Stehr, Bennet Böddecker, Daniel Kondermann, Matthias Rottmann

TL;DR

The paper tackles the challenge of obtaining high-quality soft labels with quantified epistemic uncertainty from limited crowdsourced annotations. It introduces Probabilistic Label Spreading (PLS), a scalable graph-diffusion method that propagates single annotations on a sparse $k$-NN graph to produce per-point class distributions and uncertainty estimates. The authors prove PAC-style consistency under smoothness and density assumptions and validate the approach across multiple image datasets, showing substantial annotation-budget reductions and state-of-the-art performance on a data-centric benchmark. Practically, PLS enables robust, interpretable soft labeling for large datasets, with controllable bias-variance via the spreading parameter and graph construction, and a public implementation for reproducibility.

Abstract

Safe artificial intelligence for perception tasks remains a major challenge, partly due to the lack of data with high-quality labels. Annotations themselves are subject to aleatoric and epistemic uncertainty, which is typically ignored during annotation and evaluation. While crowdsourcing enables collecting multiple annotations per image to estimate these uncertainties, this approach is impractical at scale due to the required annotation effort. We introduce a probabilistic label spreading method that provides reliable estimates of aleatoric and epistemic uncertainty of labels. Assuming label smoothness over the feature space, we propagate single annotations using a graph-based diffusion method. We prove that label spreading yields consistent probability estimators even when the number of annotations per data point converges to zero. We present and analyze a scalable implementation of our method. Experimental results indicate that, compared to baselines, our approach substantially reduces the annotation budget required to achieve a desired label quality on common image datasets and achieves a new state of the art on the Data-Centric Image Classification benchmark.

Probabilistic Label Spreading: Efficient and Consistent Estimation of Soft Labels with Epistemic Uncertainty on Graphs

TL;DR

The paper tackles the challenge of obtaining high-quality soft labels with quantified epistemic uncertainty from limited crowdsourced annotations. It introduces Probabilistic Label Spreading (PLS), a scalable graph-diffusion method that propagates single annotations on a sparse -NN graph to produce per-point class distributions and uncertainty estimates. The authors prove PAC-style consistency under smoothness and density assumptions and validate the approach across multiple image datasets, showing substantial annotation-budget reductions and state-of-the-art performance on a data-centric benchmark. Practically, PLS enables robust, interpretable soft labeling for large datasets, with controllable bias-variance via the spreading parameter and graph construction, and a public implementation for reproducibility.

Abstract

Safe artificial intelligence for perception tasks remains a major challenge, partly due to the lack of data with high-quality labels. Annotations themselves are subject to aleatoric and epistemic uncertainty, which is typically ignored during annotation and evaluation. While crowdsourcing enables collecting multiple annotations per image to estimate these uncertainties, this approach is impractical at scale due to the required annotation effort. We introduce a probabilistic label spreading method that provides reliable estimates of aleatoric and epistemic uncertainty of labels. Assuming label smoothness over the feature space, we propagate single annotations using a graph-based diffusion method. We prove that label spreading yields consistent probability estimators even when the number of annotations per data point converges to zero. We present and analyze a scalable implementation of our method. Experimental results indicate that, compared to baselines, our approach substantially reduces the annotation budget required to achieve a desired label quality on common image datasets and achieves a new state of the art on the Data-Centric Image Classification benchmark.
Paper Structure (25 sections, 7 theorems, 97 equations, 15 figures, 5 tables, 3 algorithms)

This paper contains 25 sections, 7 theorems, 97 equations, 15 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Numerical Results
  5. Limitations
  6. Conclusion
  7. Supplementary Material Overview
  8. Additional Numerical Results
  9. Details on Datasets and Simulated Soft Labels
  10. Illustration on the Two Moons Dataset
  11. Performance Comparison on Different Datasets
  12. Hyperparameter Analysis for Different Budgets
  13. Labeling Large Datasets with few Annotations
  14. Received Feedback Depending on Spreading Intensity
  15. Performance on Training vs. Test Data
  16. ...and 10 more sections

Key Result

Theorem 1

Let $\varepsilon> 0$ and $\delta \in(0,1)$. Under the above assumptions, when choosing the spreading intensity as $\alpha_n = 1 - n^{\frac{1}{d+1}} \to 1$ dependent on the dataset size, the bandwidth of the epsilon-graph as $h_n = \varepsilon / (3 L_y \lceil\log_{\alpha_n} \varepsilon / (\sqrt{2}\, with some constant $\zeta$ depending on $\varepsilon$, the minimal and maximal density $p_{\min}$,

Figures (15)

  • Figure 1: Probabilistic label spreading estimates soft labels based on few noisy annotations through graph-based propagation (a). It also estimates epistemic uncertainty of soft labels (b).
  • Figure 2: Overview of our method. Images are embedded using a feature extractor (e.g. a vision-language model combined with a dimensionality reduction technique). PLS is then applied in the embedding space to estimate soft labels for all features.
  • Figure 3: Comparison of performance trajectories on CIFAR-10-H for our method with different spreading intensities $\alpha$ (a) and, in comparison with baseline algorithms (b). With a growing number of annotations provided to the algorithm, RMSE decreases.
  • Figure 4: Confidence Intervals for the estimates of our method derived from 1) Wilson score and 2) Hoeffding type bounds.
  • Figure A.1: Distribution of the normalized entropy in the soft labels of each dataset. The histogram's values corresponds to the left y-axis and the line plot indicates the empirical cumulative distribution function using the right y-axis.
  • ...and 10 more figures

Theorems & Definitions (7)

  • Theorem 1
  • Theorem 2: Theorem 1 of the main manuscript
  • Theorem 3: Bernstein inequality for bounded distributions (Theorem 2.9.5 of vershynin_highdimprob_2026)
  • Lemma 1: Concentration of $\Phi_q^{B_\varepsilon}$
  • Theorem 4: Hoeffding's inequality for bounded random variables (Theorem 2.2.6 of vershynin_highdimprob_2026)
  • Lemma 2: Concentration of $\Phi_q$
  • Lemma 3: Concentration of $\Phi_q - \Phi_q^{B_\varepsilon}$