Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Provably-Correct and Robust Convex Model for Smooth Separable NMF

Junjun Pan, Valentin Leplat, Michael Ng, Nicolas Gillis

TL;DR

A convex model is proposed for SSNMF and it is shown that it provably recovers the sought-after factors, even in the presence of noise, and compares favorably with state-of-the-art methods on both synthetic and hyperspectral datasets.

Abstract

Nonnegative matrix factorization (NMF) is a linear dimensionality reduction technique for nonnegative data, with applications such as hyperspectral unmixing and topic modeling. NMF is a difficult problem in general (NP-hard), and its solutions are typically not unique. To address these two issues, additional constraints or assumptions are often used. In particular, separability assumes that the basis vectors in the NMF are equal to some columns of the input matrix. In that case, the problem is referred to as separable NMF (SNMF) and can be solved in polynomial-time with robustness guarantees, while identifying a unique solution. However, in real-world scenarios, due to noise or variability, multiple data points may lie near the basis vectors, which SNMF does not leverage. In this work, we rely on the smooth separability assumption, which assumes that each basis vector is close to multiple data points. We explore the properties of the corresponding problem, referred to as smooth SNMF (SSNMF), and examine how it relates to SNMF and orthogonal NMF. We then propose a convex model for SSNMF and show that it provably recovers the sought-after factors, even in the presence of noise. We finally adapt an existing fast gradient method to solve this convex model for SSNMF, and show that it compares favorably with state-of-the-art methods on both synthetic and hyperspectral datasets.

A Provably-Correct and Robust Convex Model for Smooth Separable NMF

TL;DR

A convex model is proposed for SSNMF and it is shown that it provably recovers the sought-after factors, even in the presence of noise, and compares favorably with state-of-the-art methods on both synthetic and hyperspectral datasets.

Abstract

Nonnegative matrix factorization (NMF) is a linear dimensionality reduction technique for nonnegative data, with applications such as hyperspectral unmixing and topic modeling. NMF is a difficult problem in general (NP-hard), and its solutions are typically not unique. To address these two issues, additional constraints or assumptions are often used. In particular, separability assumes that the basis vectors in the NMF are equal to some columns of the input matrix. In that case, the problem is referred to as separable NMF (SNMF) and can be solved in polynomial-time with robustness guarantees, while identifying a unique solution. However, in real-world scenarios, due to noise or variability, multiple data points may lie near the basis vectors, which SNMF does not leverage. In this work, we rely on the smooth separability assumption, which assumes that each basis vector is close to multiple data points. We explore the properties of the corresponding problem, referred to as smooth SNMF (SSNMF), and examine how it relates to SNMF and orthogonal NMF. We then propose a convex model for SSNMF and show that it provably recovers the sought-after factors, even in the presence of noise. We finally adapt an existing fast gradient method to solve this convex model for SSNMF, and show that it compares favorably with state-of-the-art methods on both synthetic and hyperspectral datasets.

Paper Structure

This paper contains 29 sections, 13 theorems, 56 equations, 10 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contribution and outline
  3. Notation
  4. Separable, Smooth and Orthogonal NMF
  5. Separable NMF (SNMF)
  6. Separability using self-dictionary and convex relaxation
  7. Smooth Separable NMF (SSNMF)
  8. Orthogonal NMF (ONMF)
  9. ONMF and self-dictionary
  10. Convex Model for SSNMF and Recovery Guarantees
  11. A Convex Model for SSNMF
  12. Uniqueness of the optimal solution of \ref{['eq:convexmodelSSNMF']}
  13. Recovery Guarantees
  14. Discussion on the Robustness Guarantees
  15. Reducing the variance to better estimate $W$
  16. ...and 14 more sections

Key Result

Theorem 1

gillis2014robust Let $M = WH+N$ satisfy Assumption assum:stocNMF and $M$ be $r$-separable, that is, $M = W[I_r \textcolor{blue}{,} H']\Pi + N$ for some permutation matrix $\Pi$. Let also $\|N\|_1 \leq \epsilon$, $\max_{k,j} H'(k,j) = \beta < 1$, $X^*$ be an optimal solution of modelsepNMF, and $\ma then $|\mathcal{K}| = r$ and

Figures (10)

  • Figure 1: Illustration of Assumption \ref{['assum:stocNMF']} in the noiseless case ($N=0$) for $r=3$.
  • Figure 2: Average (over trials) for Dirichlet mixtures: average results across trials for FGNSR, SPA, CSSNMF, SSPA(min), SSPA(mid), SSPA(mean)
  • Figure 3: Middle points (adversarial noise): average results across trials for FGNSR, SPA, CSSNMF, SSPA(min), SSPA(mid), SSPA(mean).
  • Figure 4: Outliers experiment: CSSNMF with average vs. median aggregation.
  • Figure 5: Jasper Ridge endmember spectral signatures: : ground truth (top left), CSSNMF (top right), SPA (bottom left) and SSPA with tuned nplp (bottom right).
  • ...and 5 more figures

Theorems & Definitions (24)

  • Theorem 1
  • Theorem 2
  • proof
  • Lemma 1
  • proof
  • Example 1
  • Lemma 2
  • Lemma 3
  • proof
  • Lemma 4
  • ...and 14 more