Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MM-algorithms for traditional and convex NMF with Tweedie and Negative Binomial cost functions and empirical evaluation

Elisabeth Sommer James, Asger Hobolth, Marta Pelizzola

TL;DR

A unified framework for both traditional and convex NMF under a broad class of distributional assumptions, including Negative Binomial and Tweedie models, where the connection between the Tweedie and the $\beta$-divergence is also highlighted.

Abstract

Non-negative matrix factorisation (NMF) is a widely used tool for unsupervised learning and feature extraction, with applications ranging from genomics to text analysis and signal processing. Standard formulations of NMF are typically derived under Gaussian or Poisson noise assumptions, which may be inadequate for data exhibiting overdispersion or other complex mean-variance relationships. In this paper, we develop a unified framework for both traditional and convex NMF under a broad class of distributional assumptions, including Negative Binomial and Tweedie models, where the connection between the Tweedie and the $β$-divergence is also highlighted. Using a Majorize-Minimisation approach, we derive multiplicative update rules for all considered models, and novel updates for convex NMF with Poisson and Negative Binomial cost functions. We provide a unified implementation of all considered models, including the first implementations of several convex NMF models. Empirical evaluations on mutational and word count data demonstrate that the choice of noise model critically affects model fit and feature recovery, and that convex NMF can provide an efficient and robust alternative to traditional NMF in scenarios where the number of classes is large. The code for our proposed updates is available in the R package nmfgenr and can be found at https://github.com/MartaPelizzola/nmfgenr.

MM-algorithms for traditional and convex NMF with Tweedie and Negative Binomial cost functions and empirical evaluation

TL;DR

A unified framework for both traditional and convex NMF under a broad class of distributional assumptions, including Negative Binomial and Tweedie models, where the connection between the Tweedie and the -divergence is also highlighted.

Abstract

Non-negative matrix factorisation (NMF) is a widely used tool for unsupervised learning and feature extraction, with applications ranging from genomics to text analysis and signal processing. Standard formulations of NMF are typically derived under Gaussian or Poisson noise assumptions, which may be inadequate for data exhibiting overdispersion or other complex mean-variance relationships. In this paper, we develop a unified framework for both traditional and convex NMF under a broad class of distributional assumptions, including Negative Binomial and Tweedie models, where the connection between the Tweedie and the -divergence is also highlighted. Using a Majorize-Minimisation approach, we derive multiplicative update rules for all considered models, and novel updates for convex NMF with Poisson and Negative Binomial cost functions. We provide a unified implementation of all considered models, including the first implementations of several convex NMF models. Empirical evaluations on mutational and word count data demonstrate that the choice of noise model critically affects model fit and feature recovery, and that convex NMF can provide an efficient and robust alternative to traditional NMF in scenarios where the number of classes is large. The code for our proposed updates is available in the R package nmfgenr and can be found at https://github.com/MartaPelizzola/nmfgenr.
Paper Structure (18 sections, 31 equations, 12 figures, 4 tables, 1 algorithm)

This paper contains 18 sections, 31 equations, 12 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methods
  3. Notation
  4. Tweedie distribution
  5. Special cases: Normal and Poisson distribution
  6. Negative Binomial distribution
  7. Relation between Negative Binomial and Tweedie NMF methods
  8. Computational aspects
  9. Results
  10. Mutational count data from liver cancer patients
  11. Topic modelling of newsgroups data
  12. Discussion
  13. Power fitting for Tweedie NMF
  14. Fitting of dispersion parameter for Negative Binomial NMF
  15. Derivation of updates for Negative Binomial convex NMF
  16. ...and 3 more sections

Figures (12)

  • Figure 1: Unit deviances and the corresponding Tweedie distributions with parameters $(\mu, \sigma^2) = (50,1)$ for $p \in \{0,1\}$, $p \in \{1.5,2\}$, and $p \in \{2.5,3\}$.
  • Figure 2: Run time analysis per iteration. The run time in seconds is plotted against the number of data points $N$ for the different methods and distributional assumptions and number of dimensions M. Results are averaged over 100 iterations. The rank of the factorisation is K = 5 as estimated in Section \ref{['sec:liver']}, and the values of the dispersion parameter $\alpha$ are as given in \ref{['fig:residualplotliver']}, being 45.21 and 36.44 for traditional and convex NMF respectively.
  • Figure 3: Comparison of BIC values and mean-variance relationships across the 8 NMF models, applied to the mutational count data of liver cancer patients.
  • Figure 4: Residual analysis of NMF/$\mathcal{T}$ (on the left) and NMF/$\mathcal{C}$ (on the right) under the Normal, Poisson, Tweedie, and Negative Binomial distributions applied to the mutational count data of liver cancer patients. In each panel, the coloured lines are 2 standard deviations of the assumed model.
  • Figure 5: Quality of estimated signatures for mutational count data of liver cancer patients. Each method is compared to NMF/$\mathcal{T}$/NB$_{45.21}$/5 (left) and to the COSMIC signatures (right). We show the quality of the estimated signatures measured by cosine similarity for all methods.
  • ...and 7 more figures