A Latent-Variable Formulation of the Poisson Canonical Polyadic Tensor Model: Maximum Likelihood Estimation and Fisher Information

TL;DR

This work reframes the Poisson canonical polyadic (PCP) tensor model for count data as a latent-variable model, enabling classical likelihood-based inference via EM-type algorithms. By introducing a complete loglikelihood on a higher-dimensional latent tensor, the authors show that many existing factorization methods (e.g., NMF and CP-APR) arise as instances of EM/ECM/MCECM, and they derive both the observed and expected Fisher information via Oakes’ theorem and the missing-information principle. They provide a closed-form, rank-one solution with provable identifiability properties and furnish general results for the Fisher information in the multi-way setting, including a conjectured rank relation that elucidates identifiability and redundancy in CP-like decompositions. Numerical experiments validate the Fisher-information expressions and support the rank-conjecture across various tensor orders, sizes, and ranks. The latent-variable approach thus yields concrete tools for parameter inference, rank selection, and diagnostic capabilities in counting-tensor applications such as networks, text mining, and geospatial data analysis.

Abstract

We establish parameter inference for the Poisson canonical polyadic (PCP) model of tensor count data through a latent-variable formulation. Our approach exploits the property that any random tensor that follows the PCP model can be derived by marginalizing an unobservable random tensor of one dimension larger. The loglikelihood of this larger dimensional tensor, referred to as the "complete" loglikelihood, is comprised of multiple loglikelihoods corresponding to rank one PCP models. Using this methodology, we first demonstrate that several existing algorithms for fitting non-negative matrix and tensor factorizations are Expectation-Maximization algorithms. Next, we derive the observed and expected Fisher information matrices for the PCP model by leveraging its latent-variable formulation. The Fisher information provides us crucial insights into the well-posedness of the tensor model, such as the role that the rank of parameter tensor plays in identifiability and indeterminacy. For the special case of PCP models with rank one parameter tensors, we demonstrate that these results are greatly simplified.

Figures (3)

• Figure 1: Boxplots of relative error $||\mathcal{I} (\boldsymbol{\theta}) - \widehat{\mathcal{I} }_K(\boldsymbol{\theta})||_F/||\mathcal{I} (\boldsymbol{\theta})||_F$ between the Monte-Carlo estimated $\widehat{\mathcal{I} }_K(\boldsymbol{\theta})$ and our analytical expression for $\mathcal{I} (\boldsymbol{\theta})$ in Theorem \ref{['thm:OFIM_genR']}, across different values of $N,S,R,K$.
• Figure 2: (a) Comparison of numerical and conjectured Fisher information matrix ranks across various values of $P$, $N$ and $R$. The numerical and conjectured ranks are the same in all cases. (b) Comparison of the numerical Fisher information matrix rank against $N^P$ (in green) and $PNR-L$ (in orange) when $P=3$ and $N=8$ are fixed, but the CP rank increases $R=1,2,\dots,46$ . According to our conjecture, we should expect a flat orange line in one when $R<23$ and a flat green line in one when $R>23$.
• Figure :

Theorems & Definitions (23)

• Lemma 1
• proof
• Theorem 1
• proof
• Corollary 1
• proof
• Theorem 2: oakes99
• Theorem 3
• proof
• Conjecture 1