Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Revisiting Sparsity Constraint Under High-Rank Property in Partial Multi-Label Learning

Chongjie Si, Yidan Cui, Fuchao Yang, Xiaokang Yang, Wei Shen

TL;DR

This work revisits Partial Multi-Label Learning (PML) by challenging the common assumption that noise is sparse and true labels are low-rank; real-world label matrices are typically full-rank, creating a conflict in prior methods. The authors propose Schirn, a method that enforces sparsity on the noise label matrix while preserving a high-rank property for the predicted label matrix, achieved via a convex-relaxed objective that combines a nuclear-norm term and an L1 penalty, solved with an Augmented Lagrangian framework. Theoretical support is provided by a rank-perturbation argument showing that sparse noise preserves high rank under perturbations, and extensive experiments on 11 datasets (5 real-world, 6 synthetic) demonstrate that Schirn outperforms state-of-the-art PML methods across multiple metrics. The work introduces a principled perspective on the sparsity-rank relationship in PML, with practical impact on disambiguating noisy candidate labels while maintaining rich label structure; future work includes extending Schirn to end-to-end deep learning architectures.

Abstract

Partial Multi-Label Learning (PML) extends the multi-label learning paradigm to scenarios where each sample is associated with a candidate label set containing both ground-truth labels and noisy labels. Existing PML methods commonly rely on two assumptions: sparsity of the noise label matrix and low-rankness of the ground-truth label matrix. However, these assumptions are inherently conflicting and impractical for real-world scenarios, where the true label matrix is typically full-rank or close to full-rank. To address these limitations, we demonstrate that the sparsity constraint contributes to the high-rank property of the predicted label matrix. Based on this, we propose a novel method Schirn, which introduces a sparsity constraint on the noise label matrix while enforcing a high-rank property on the predicted label matrix. Extensive experiments demonstrate the superior performance of Schirn compared to state-of-the-art methods, validating its effectiveness in tackling real-world PML challenges.

Revisiting Sparsity Constraint Under High-Rank Property in Partial Multi-Label Learning

TL;DR

This work revisits Partial Multi-Label Learning (PML) by challenging the common assumption that noise is sparse and true labels are low-rank; real-world label matrices are typically full-rank, creating a conflict in prior methods. The authors propose Schirn, a method that enforces sparsity on the noise label matrix while preserving a high-rank property for the predicted label matrix, achieved via a convex-relaxed objective that combines a nuclear-norm term and an L1 penalty, solved with an Augmented Lagrangian framework. Theoretical support is provided by a rank-perturbation argument showing that sparse noise preserves high rank under perturbations, and extensive experiments on 11 datasets (5 real-world, 6 synthetic) demonstrate that Schirn outperforms state-of-the-art PML methods across multiple metrics. The work introduces a principled perspective on the sparsity-rank relationship in PML, with practical impact on disambiguating noisy candidate labels while maintaining rich label structure; future work includes extending Schirn to end-to-end deep learning architectures.

Abstract

Partial Multi-Label Learning (PML) extends the multi-label learning paradigm to scenarios where each sample is associated with a candidate label set containing both ground-truth labels and noisy labels. Existing PML methods commonly rely on two assumptions: sparsity of the noise label matrix and low-rankness of the ground-truth label matrix. However, these assumptions are inherently conflicting and impractical for real-world scenarios, where the true label matrix is typically full-rank or close to full-rank. To address these limitations, we demonstrate that the sparsity constraint contributes to the high-rank property of the predicted label matrix. Based on this, we propose a novel method Schirn, which introduces a sparsity constraint on the noise label matrix while enforcing a high-rank property on the predicted label matrix. Extensive experiments demonstrate the superior performance of Schirn compared to state-of-the-art methods, validating its effectiveness in tackling real-world PML challenges.

Paper Structure

This paper contains 28 sections, 1 theorem, 20 equations, 2 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multi-Label Learning (MLL)
  4. Partial Multi-label Learning (PML)
  5. Proposed Approach
  6. Notation
  7. Ordinary Classifier
  8. Sparsity Constraint Under High-Rank Property
  9. Overall Formulation
  10. Optimization
  11. $\mathbf{W}$ Subproblem
  12. $\mathbf{N}$ Subproblem
  13. $\mathbf{C}$ Subproblem
  14. Lagrange Term Subproblem
  15. Experiments
  16. ...and 13 more sections

Key Result

Theorem 3.1

Let $\mathbf{Y} \in \mathbb{R}^{n \times l}$ be a full-rank matrix with $\text{rank}(\mathbf{Y}) = \min(n, l)$, and let $\mathbf{N} \in \mathbb{R}^{n \times l}$ be a sparse binary matrix satisfying $\|\mathbf{N}\|_0 \leq \epsilon$, where $\|\cdot\|_0$ represents the $\ell_0$-norm and $\epsilon$ is a where $\Delta$ is a very small positive integer that depends on the sparsity level $\epsilon$ of $\

Figures (2)

  • Figure 1: An example of partial multi-label learning.
  • Figure 2: Sensitivity analysis of Schirn.

Theorems & Definitions (1)

  • Theorem 3.1