Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Iterative Algorithm for Regularized Non-negative Matrix Factorizations

Steven E. Pav

TL;DR

The companion R package rnnmf is applied to the problem of finding a reduced rank representation of a database of cocktails, and the non-negative matrix factorization algorithm of Lee and Seung is generalized to accept a weighted norm.

Abstract

We generalize the non-negative matrix factorization algorithm of Lee and Seung to accept a weighted norm, and to support ridge and Lasso regularization. We recast the Lee and Seung multiplicative update as an additive update which does not get stuck on zero values. We apply the companion R package rnnmf to the problem of finding a reduced rank representation of a database of cocktails.

An Iterative Algorithm for Regularized Non-negative Matrix Factorizations

TL;DR

The companion R package rnnmf is applied to the problem of finding a reduced rank representation of a database of cocktails, and the non-negative matrix factorization algorithm of Lee and Seung is generalized to accept a weighted norm.

Abstract

We generalize the non-negative matrix factorization algorithm of Lee and Seung to accept a weighted norm, and to support ridge and Lasso regularization. We recast the Lee and Seung multiplicative update as an additive update which does not get stuck on zero values. We apply the companion R package rnnmf to the problem of finding a reduced rank representation of a database of cocktails.

Paper Structure

This paper contains 10 sections, 4 theorems, 34 equations, 6 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Regularized Non-Negative Matrix Factorization
  3. Split Form
  4. Factorization in Vector Form
  5. As a Matrix Factorization Algorithm
  6. Additive Steps and Convergence
  7. Other Directions
  8. Simulations
  9. Applications
  10. Matrix Identities

Key Result

Lemma 2.1

Let ${\mathsf{G}\xspace}^{{}}_{}$ be a symmetric matrix with non-negative elements, and full rank, and let ${\boldsymbol{b}\xspace}^{{}}_{}$ be a vector with strictly positive elements. Then (Here $\mathsf{A}\xspace\succeq\mathsf{B}\xspace$ means that $\mathsf{A}\xspace - \mathsf{B}\xspace$ is positive semidefinite.)

Figures (6)

  • Figure 1: The Frobenius norm is plotted versus step for two methods for a small problem.
  • Figure 2: The Frobenius norm is plotted versus step for two methods for a small problem. Starting iterates are taken to be sparse or dense.
  • Figure 3: $R^2$ is plotted versus $d$ for the unregularized factorization of the cocktail data.
  • Figure 4: The first three latent cocktails are shown. Ingredients making less than 0.03 of the total are lumped together. Patterns are courtesy of the ggpattern package. ggpattern
  • Figure 5: The second fit of three latent cocktails are shown. Ingredients making less than 0.03 of the total lumped together.
  • ...and 1 more figures

Theorems & Definitions (8)

  • Lemma 2.1
  • proof
  • Lemma 2.2
  • proof
  • Theorem 2.3: LeeSeung
  • proof
  • Lemma 3.1
  • proof