Amortized Variational Inference for Partial-Label Learning: A Probabilistic Approach to Label Disambiguation
Tobias Fuchs, Nadja Klein
TL;DR
The paper tackles learning from partially labeled data by directly estimating the posterior over true labels using amortized variational inference. It introduces ViPll, which uses a neural encoder to predict Dirichlet variational parameters $\alpha_\phi$ for $Q_\phi(Y|X,S)$ and a CVAE-based generator $p_{\theta,\gamma}(X|Y)$ to model feature distributions, all optimized via a ($\beta$-weighted) ELBO. A maximum-entropy prior $p(y)$ under candidate-set constraints and a fixed-form candidate-set likelihood $p(s|x,y)$ regularize learning, with a three-phase training procedure and Monte Carlo sampling to scale efficiently. Empirical results on real PLL and synthetic settings show state-of-the-art accuracy and favorable efficiency, supported by ablations that validate the utility of the generative model and the Markov-equivalent causal formulation. The work offers a principled, architecture-agnostic approach that integrates deep learning with rigorous probabilistic modeling for disambiguating noisy labels in practical applications.
Abstract
Real-world data is frequently noisy and ambiguous. In crowdsourcing, for example, human annotators may assign conflicting class labels to the same instances. Partial-label learning (PLL) addresses this challenge by training classifiers when each instance is associated with a set of candidate labels, only one of which is correct. While early PLL methods approximate the true label posterior, they are often computationally intensive. Recent deep learning approaches improve scalability but rely on surrogate losses and heuristic label refinement. We introduce a novel probabilistic framework that directly approximates the posterior distribution over true labels using amortized variational inference. Our method employs neural networks to predict variational parameters from input data, enabling efficient inference. This approach combines the expressiveness of deep learning with the rigor of probabilistic modeling, while remaining architecture-agnostic. Theoretical analysis and extensive experiments on synthetic and real-world datasets demonstrate that our method achieves state-of-the-art performance in both accuracy and efficiency.