Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inaccurate Label Distribution Learning with Dependency Noise

Zhiqiang Kou, Jing Wang, Yuheng Jia, Xin Geng

TL;DR

The paper tackles unstable label distributions caused by instance- and label-dependent noise in LDL by proposing DN-ILDL, which models the noisy distribution as $oldsymbol{oldsymbol{}}=oldsymbol{D}+oldsymbol{E}$ with $oldsymbol{E}=oldsymbol{X}oldsymbol{P}+oldsymbol{Y}oldsymbol{Q}$ and learns a low-rank mapping $oldsymbol{W}$ from features to true distributions. It jointly optimizes $oldsymbol{W}$, $oldsymbol{P}$, and $oldsymbol{Q}$ under a nuclear-norm and group-sparsity regularization, while enforcing graph-regularized alignment between input and output topologies via $oldsymbol{S}$ and $ ilde{oldsymbol{S}}=oldsymbol{ extΦ}(oldsymbol{W},oldsymbol{X},oldsymbol{ abla})$; optimization is performed with ADMM, and theoretical recovery and generalization bounds are established. The method demonstrates strong empirical performance across 13 real-world datasets, outperforming six LDL baselines and one ILDL method, and shows robustness to parameter choices. The work advances LDL by explicitly modeling dependent noise and integrating topology-preserving constraints, with practical impact for noisy annotation scenarios in diverse domains.

Abstract

In this paper, we introduce the Dependent Noise-based Inaccurate Label Distribution Learning (DN-ILDL) framework to tackle the challenges posed by noise in label distribution learning, which arise from dependencies on instances and labels. We start by modeling the inaccurate label distribution matrix as a combination of the true label distribution and a noise matrix influenced by specific instances and labels. To address this, we develop a linear mapping from instances to their true label distributions, incorporating label correlations, and decompose the noise matrix using feature and label representations, applying group sparsity constraints to accurately capture the noise. Furthermore, we employ graph regularization to align the topological structures of the input and output spaces, ensuring accurate reconstruction of the true label distribution matrix. Utilizing the Alternating Direction Method of Multipliers (ADMM) for efficient optimization, we validate our method's capability to recover true labels accurately and establish a generalization error bound. Extensive experiments demonstrate that DN-ILDL effectively addresses the ILDL problem and outperforms existing LDL methods.

Inaccurate Label Distribution Learning with Dependency Noise

TL;DR

The paper tackles unstable label distributions caused by instance- and label-dependent noise in LDL by proposing DN-ILDL, which models the noisy distribution as with and learns a low-rank mapping from features to true distributions. It jointly optimizes , , and under a nuclear-norm and group-sparsity regularization, while enforcing graph-regularized alignment between input and output topologies via and ; optimization is performed with ADMM, and theoretical recovery and generalization bounds are established. The method demonstrates strong empirical performance across 13 real-world datasets, outperforming six LDL baselines and one ILDL method, and shows robustness to parameter choices. The work advances LDL by explicitly modeling dependent noise and integrating topology-preserving constraints, with practical impact for noisy annotation scenarios in diverse domains.

Abstract

In this paper, we introduce the Dependent Noise-based Inaccurate Label Distribution Learning (DN-ILDL) framework to tackle the challenges posed by noise in label distribution learning, which arise from dependencies on instances and labels. We start by modeling the inaccurate label distribution matrix as a combination of the true label distribution and a noise matrix influenced by specific instances and labels. To address this, we develop a linear mapping from instances to their true label distributions, incorporating label correlations, and decompose the noise matrix using feature and label representations, applying group sparsity constraints to accurately capture the noise. Furthermore, we employ graph regularization to align the topological structures of the input and output spaces, ensuring accurate reconstruction of the true label distribution matrix. Utilizing the Alternating Direction Method of Multipliers (ADMM) for efficient optimization, we validate our method's capability to recover true labels accurately and establish a generalization error bound. Extensive experiments demonstrate that DN-ILDL effectively addresses the ILDL problem and outperforms existing LDL methods.
Paper Structure (12 sections, 2 theorems, 13 equations, 5 figures, 4 tables, 1 algorithm)

This paper contains 12 sections, 2 theorems, 13 equations, 5 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. The DI-ILDL Approach
  4. Algorithm
  5. Optimizing using ADMM
  6. The Complexity Analysis
  7. Theoretical Analysis
  8. Experiments
  9. Datasets and Evaluation Metrics
  10. Comparative Studies
  11. Further Analysis
  12. Conclusion

Key Result

Theorem 1

Assume the actual noise matrices $\mathbf{E}^{*}$ depend on both instance and label characteristics, exhibiting group sparsity as indicated by sparsity levels $S_x$ and $S_y$, and group counts $G_x$ and $G_y$. With $\mathbf{W}$ fixed in Equation $(\mathcal{L})$, we consider $\mathbf{E}^{'} = \mathbf with a probability exceeding $1 - 2/g^2$. This result suggests that our algorithm is likely to conv

Figures (5)

  • Figure 1: An image from a natural-scene dataset geng2014multilabel with a label distribution.
  • Figure 2: CD diagrams of the comparing algorithms in terms of each evaluation criterion. For the tests, CD equals 2.3296 at 0.05 significance level.
  • Figure 3: Ablation results on seven datasets in terms of Clark $\downarrow$, KL $\downarrow$.
  • Figure 4: The performance of DI-ILDL with $\alpha$, $\beta$ and $\gamma$ varying from $\left\{0.005, 0.01, 0.05, 0.1, 0.5, 1, 10\right\}$ in terms of KL-ditance on five datasets.
  • Figure 5: Convergence of the objective functions of Eq. (\ref{['finall loss']}) with respect to thenumber of iterations on (a) Natural-Scene and (b) Yeast-heat.

Theorems & Definitions (2)

  • Theorem 1
  • Theorem 2