Latent label distribution grid representation for modeling uncertainty

TL;DR

LDL can capture polysemy but annotation noise induces uncertainty in the label space, harming learning. The paper presents Latent Label Distribution Grid (LLDG), which builds a label correlation grid from label differences and expands entries into Gaussian components to form a latent 3D grid, then uses Tucker-based low-rank regularization to suppress noise; a local-global feature extractor produces the grid and an LLDG-Mixer reconstructs the final label distribution with a combined loss. Key contributions include integrating correlation-based representations with probabilistic expansion, introducing Tucker-based noise reduction, and demonstrating the approach on 15 LDL datasets plus a MedMNIST transfer, with strong robustness to noise. The work offers a principled, scalable framework for uncertainty-aware LDL and extends naturally to downstream classification tasks. Overall, LLDG provides improved stability and performance by explicitly modeling label-space uncertainty through a latent grid structure.

Abstract

Although \textbf{L}abel \textbf{D}istribution \textbf{L}earning (LDL) has promising representation capabilities for characterizing the polysemy of an instance, the complexity and high cost of the label distribution annotation lead to inexact in the construction of the label space. The existence of a large number of inexact labels generates a label space with uncertainty, which misleads the LDL algorithm to yield incorrect decisions. To alleviate this problem, we model the uncertainty of label distributions by constructing a \textbf{L}atent \textbf{L}abel \textbf{D}istribution \textbf{G}rid (LLDG) to form a low-noise representation space. Specifically, we first construct a label correlation matrix based on the differences between labels, and then expand each value of the matrix into a vector that obeys a Gaussian distribution, thus building a LLDG to model the uncertainty of the label space. Finally, the LLDG is reconstructed by the LLDG-Mixer to generate an accurate label distribution. Note that we enforce a customized low-rank scheme on this grid, which assumes that the label relations may be noisy and it needs to perform noise-reduction with the help of a Tucker reconstruction technique. Furthermore, we attempt to evaluate the effectiveness of the LLDG by considering its generation as an upstream task to achieve the classification of the objects. Extensive experimental results show that our approach performs competitively on several benchmarks.

Figures (5)

• Figure 1: LLDG vs. label distribution matrix (LDM). This figure shows the representation patterns of LDM and LLDG, where LDM considers only the distribution of the values of the label distribution and LLDG considers the relational distribution of the values of the label distribution.
• Figure 2: Our architecture. In general, our model has two learning targets, on the one hand, to learn the label distribution, and on the other hand to learn the LLDG. Specifically, first, the input features are extracted by a local-global feature extractor to create a grid. LLDG establishes a labeled correlation space that is constrained by the Tucker reconstruction algorithm. Then, this grid is conducted in LLDG-Mixer to form a vector by squeezing. Finally, this vector is normalized by Softmax to form a label distribution.
• Figure 3: The 15 datasets include detailed statistics of instances, features, and labels.
• Figure 4: Evaluation metrics for LDL algorithms, where $\uparrow$ and $\downarrow$ represents performance favorites.
• Figure 5: This figure shows the energy distribution of the LLDG (4), and the blue boxes indicate the regions where the energy varies with the enhancement of Gaussian noise; in general, the energy of the LLDG does not vary significantly with the increase of Gaussian noise.