Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Regularized Sparse $L_{2,1}$ Semi-Nonnegative Matrix Factorization for Data Reduction

Anthony Rhodes, Bin Jiang, Jenny Jiang

TL;DR

A new L2,1 L2,1 SNF algorithm that utilizes the noise‐insensitive L2,1$$ {L}_{2,1} $$ norm and provides monotonic convergence analysis of the L2,1$$ {L}_{2,1} $$ SNF algorithm.

Abstract

Non-negative Matrix Factorization (NMF) is an effective algorithm for multivariate data analysis, including applications to feature selection, pattern recognition, and computer vision. Its variant, Semi-Nonnegative Matrix Factorization (SNF), extends the ability of NMF to render parts-based data representations to include mixed-sign data. Graph Regularized SNF builds upon this paradigm by adding a graph regularization term to preserve the local geometrical structure of the data space. Despite their successes, SNF-related algorithms to date still suffer from instability caused by the Frobenius norm due to the effects of outliers and noise. In this paper, we present a new $L_{2,1}$ SNF algorithm that utilizes the noise-insensitive $L_{2,1}$ norm. We provide monotonic convergence analysis of the $L_{2,1}$ SNF algorithm. In addition, we conduct numerical experiments on three benchmark mixed-sign datasets as well as several randomized mixed-sign matrices to demonstrate the performance superiority of $L_{2,1}$ SNF over conventional SNF algorithms under the influence of Gaussian noise at different levels.

Graph Regularized Sparse $L_{2,1}$ Semi-Nonnegative Matrix Factorization for Data Reduction

TL;DR

A new L2,1 L2,1 SNF algorithm that utilizes the noise‐insensitive L2,1 norm and provides monotonic convergence analysis of the L2,1 SNF algorithm.

Abstract

Non-negative Matrix Factorization (NMF) is an effective algorithm for multivariate data analysis, including applications to feature selection, pattern recognition, and computer vision. Its variant, Semi-Nonnegative Matrix Factorization (SNF), extends the ability of NMF to render parts-based data representations to include mixed-sign data. Graph Regularized SNF builds upon this paradigm by adding a graph regularization term to preserve the local geometrical structure of the data space. Despite their successes, SNF-related algorithms to date still suffer from instability caused by the Frobenius norm due to the effects of outliers and noise. In this paper, we present a new SNF algorithm that utilizes the noise-insensitive norm. We provide monotonic convergence analysis of the SNF algorithm. In addition, we conduct numerical experiments on three benchmark mixed-sign datasets as well as several randomized mixed-sign matrices to demonstrate the performance superiority of SNF over conventional SNF algorithms under the influence of Gaussian noise at different levels.

Paper Structure

This paper contains 21 sections, 15 theorems, 77 equations, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Algorithm Construction and Complexity Analysis
  3. NMF
  4. SNF
  5. Graph Regularized SNF
  6. $L_{2,1}$ SNF
  7. Computational Complexity Analysis
  8. Algorithm Convergence Analysis
  9. Monotonic Decrease of $\mathscr{J}(\mathbf{U},\mathbf{V})$ for $\mathbf{U}$ update with fixed $\mathbf{V}$
  10. Monotonic Decrease of $\mathscr{J}(\mathbf{U},\mathbf{V})$ for $\mathbf{V}$ update with fixed $\mathbf{U}$
  11. Monotonic Decrease of $\mathscr{J}(\mathbf{U},\mathbf{V})$ for $L_{2,1}$ SNF Algorithm
  12. Optimality of $L_{2,1}$ SNF Formula based on Convexity and KKT Condition
  13. Experimental Results
  14. Experimental Setting
  15. Performance Comparison of All Algorithms
  16. ...and 6 more sections

Key Result

Lemma 3.1

Let $\mathbf{U}(t)$ and $\mathbf{U}(t+1)$ represent consecutive updates for $\mathbf{U}$ as prescribed by wtplus1. Then $\mathscr{L}(\mathbf{U}(t+1),\mathbf{V}(t)) \leq \mathscr{L}(\mathbf{U}(t),\mathbf{V}(t))$, that is,

Figures (8)

  • Figure 1: Clustering Performance with $\sigma$ noise level on Ionosphere.
  • Figure 2: Clustering Performance with $\sigma$ noise level on Waveform.
  • Figure 3: Clustering Performance with $\sigma$ noise level on USPST.
  • Figure 4: Images of $15$ randomly selected numbers reconstructed by all SNF algorithms on USPST. Top: SNF; Second from Top: GR SNF; Second from Bottom: $L_{2,1}$ SNF; Bottom: Original image with noise where the true label is shown in red at the lower left corner.
  • Figure 5: Convergence analysis of $L_{2,1}$ SNF on: (a) Ionosphere; (b) Waveform; and (c) USPST. The y-axis for the objective function value is in the log scale.
  • ...and 3 more figures

Theorems & Definitions (30)

  • Lemma 3.1
  • proof
  • Lemma 3.2
  • proof
  • Lemma 3.3
  • proof
  • Theorem 3.4
  • proof
  • Lemma 3.5
  • proof
  • ...and 20 more