A review of NMF, PLSA, LBA, EMA, and LCA with a focus on the identifiability issue

TL;DR

The paper establishes a formal connection between five latent-factor models (NMF, LBA, EMA, PLSA, LCA) for compositional data, showing that LBA/EMA/PLSA align with LCA and symmetric PLSA while NMF provides a more general framework. It proves a central identifiability result: the solution to NMF is unique if and only if the corresponding LBA/EMA/PLSA solution is unique, allowing identifiability insights to transfer across models. The authors synthesize existing identifiability results, highlighting separability and minimum-volume conditions for NMF and inner/outer extreme solutions for LBA/EMA/PLSA, and discuss practical estimation algorithms (EM, CWLS, Metropolis, minimum-volume objectives) with references to software implementations. An illustrative time-budget dataset demonstrates how the models recover interpretable latent components, and the discussion situates these models among related approaches such as archetypal analysis. Overall, the work bridges communities, clarifies identifiability conditions, and provides a concise algorithmic guide for practitioners across domains.

Abstract

Across fields such as machine learning, social science, geography, considerable attention has been given to models that factorize a nonnegative matrix into the product of two or three matrices, subject to nonnegative or row-sum-to-1 constraints. Although these models are to a large extend similar or even equivalent, they are presented under different names, and their similarity is not well known. This paper highlights similarities among five popular models, latent budget analysis (LBA), latent class analysis (LCA), end-member analysis (EMA), probabilistic latent semantic analysis (PLSA), and nonnegative matrix factorization (NMF). We focus on an essential issue-identifiability-of these models and prove that the solution of LBA, EMA, LCA, PLSA is unique if and only if the solution of NMF is unique. We also provide a brief review for algorithms of these models. We illustrate the models with a time budget dataset from social science, and end the paper with a discussion of closely related models such as archetypal analysis.

Figures (5)

• Figure 1: Geometric illustration that the solutions of NMF, LBA, EMA, and asymmetric PLSA are not unique using Table \ref{['T: healthgenderoriginal']} and Table \ref{['T: healthgendersumto1']} where rows in Table \ref{['T: healthgenderoriginal']} and Table \ref{['T: healthgendersumto1']} are in blue and two possible basis matrices are in green and in red.
• Figure 2: Geometric illustration that the solutions of NMF, LBA, EMA, and asymmetric PLSA are not unique using Table \ref{['T: educationreadshiporiginal']} and Table \ref{['T: educationreadshipsumto1']} where rows in Table \ref{['T: educationreadshiporiginal']} and Table \ref{['T: educationreadshipsumto1']} are in blue and two possible basis matrices are in green and in red.
• Figure 3: Geometric illustrations of (a) separability of a matrix $\bm{M}$, (b) SSC of a matrix $\bm{M}^T$ by assuming that viewer stands in the nonnegative orthant, faces the origin, and looks at the two-dimensional plane $\bm{x1} = \bm{1}$gillis2020nonnegative. The blue dots "o" are rows of $\bm{M}$; the red crosses "X" are standard basis vectors $\bm{e}_1, \bm{e}_2$, $\bm{e}_3$.
• Figure 4: Simplex about time budget dataset for NMF under dimensionality $K = 3$ about rows.
• Figure 5: Simplex about time budget dataset for NMF under dimensionality $K = 3$ about columns.

Theorems & Definitions (14)

• Theorem 1
• proof
• Theorem 2
• proof
• Theorem 3
• proof
• Definition 1
• Definition 2
• Theorem 4
• proof