Efficient algorithms for regularized Poisson Non-negative Matrix Factorization

TL;DR

This work addresses regularized Poisson NMF with KL loss, where the non-Lipschitz nature of the Poisson term hinders standard gradient methods. It introduces a Block Successive Upper Minimization (BSUM) framework with carefully crafted majorizing surrogates for Lipschitz, relatively smooth, and concave regularizers, plus linear constraints, yielding two algorithms: Multiplicative Update (MU) and Quadratic Update (QU). The authors prove convergence to a coordinatewise minimum (a stationary point under regularity) and analyze the trade-offs between tight majorizing functions and convergence speed, including linesearch. Numerical experiments demonstrate that QU can converge faster per iteration than MU, while linesearch can accelerate but sometimes destabilize, and show overall linear per-iteration complexity with a dichotomy-based simplex projection. This work significantly broadens Poisson NMF applicability by enabling efficient optimization under general regularizers and simplex-type constraints, with practical impact in areas like hyperspectral imaging and physical linear unmixing.

Abstract

We consider the problem of regularized Poisson Non-negative Matrix Factorization (NMF) problem, encompassing various regularization terms such as Lipschitz and relatively smooth functions, alongside linear constraints. This problem holds significant relevance in numerous Machine Learning applications, particularly within the domain of physical linear unmixing problems. A notable challenge arises from the main loss term in the Poisson NMF problem being a KL divergence, which is non-Lipschitz, rendering traditional gradient descent-based approaches inefficient. In this contribution, we explore the utilization of Block Successive Upper Minimization (BSUM) to overcome this challenge. We build approriate majorizing function for Lipschitz and relatively smooth functions, and show how to introduce linear constraints into the problem. This results in the development of two novel algorithms for regularized Poisson NMF. We conduct numerical simulations to showcase the effectiveness of our approach.

Figures (4)

• Figure 1: (a) A function where the point $(0,0)$ is a coordinatewise minimum along the $x$ and $y$ directions but is not a local minimum. (b) The blue line represents the set of all global minima for the one-dimensional case of \ref{['eq:general-poisson-loss']}, where all coordinatewise minima are also stationary points. (c) We show the effect of adding a quadratic regularization term to \ref{['eq:general-poisson-loss']}, ensuring the uniqueness of the global minimum.
• Figure 2: Two first-order majorizing functions for the function $f(x_0,x_1) = \log(0.2x_0 + 0.8x_1)$ are considered. We observe that Lemma (\ref{['lem:EM-majorization']}) provides a tighter majorization than Lemma (\ref{['lem:bregman-majorization']}) for $f$, resulting in a more optimal update $\bm{x}^{t+1}$.
• Figure 3: Convergence curves for 1000 iterations. We remove the minimum loss $\mathcal{L}\left(\dot{\bm{W}},\dot{\bm{H}}\right)$
• Figure 4: Execution time for 100 iterations for different problem sizes. Here we fix the dimension of $\bm{H}$ to $3\times64^{2}$ and varies $\bm{W}$ from $25\times3$ to $1000\times3$.

Theorems & Definitions (29)

• definition 1: Directional derivative
• definition 2: Coordinatewise Minimum
• definition 3: Stationary Points of a function
• definition 4: Regularity of a function at a point
• lemma 1
• definition 5
• lemma 2
• proof
• theorem 1: Convergence of TBSUM Algorithm \ref{['alg:TBSUM']}
• lemma 3: Log majorization