Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Probabilistic PCA on tensors

Yaoming Zhen, Piotr Zwiernik

TL;DR

This work generalizes PPCA to tensors by constraining the loading operator to have Tucker structure, yielding a probabilistic multilinear PCA model that enables uncertainty quantification and naturally accommodates multiple, possibly heterogeneous, tensor observations.

Abstract

In probabilistic principal component analysis (PPCA), an observed vector is modeled as a linear transformation of a low-dimensional Gaussian factor plus isotropic noise. We generalize PPCA to tensors by constraining the loading operator to have Tucker structure, yielding a probabilistic multilinear PCA model that enables uncertainty quantification and naturally accommodates multiple, possibly heterogeneous, tensor observations. We develop the associated theory: we establish identifiability of the loadings and noise variance and show that-unlike in matrix PPCA-the maximum likelihood estimator (MLE) exists even from a single tensor sample. We then study two estimators. First, we consider the MLE and propose an expectation maximization (EM) algorithm to compute it. Second, exploiting that Tucker maps correspond to rank-one elements after a Kronecker lifting, we design a computationally efficient estimator for which we provide provable finite-sample guarantees. Together, these results provide a coherent probabilistic framework and practical algorithms for learning from tensor-valued data.

Probabilistic PCA on tensors

TL;DR

This work generalizes PPCA to tensors by constraining the loading operator to have Tucker structure, yielding a probabilistic multilinear PCA model that enables uncertainty quantification and naturally accommodates multiple, possibly heterogeneous, tensor observations.

Abstract

In probabilistic principal component analysis (PPCA), an observed vector is modeled as a linear transformation of a low-dimensional Gaussian factor plus isotropic noise. We generalize PPCA to tensors by constraining the loading operator to have Tucker structure, yielding a probabilistic multilinear PCA model that enables uncertainty quantification and naturally accommodates multiple, possibly heterogeneous, tensor observations. We develop the associated theory: we establish identifiability of the loadings and noise variance and show that-unlike in matrix PPCA-the maximum likelihood estimator (MLE) exists even from a single tensor sample. We then study two estimators. First, we consider the MLE and propose an expectation maximization (EM) algorithm to compute it. Second, exploiting that Tucker maps correspond to rank-one elements after a Kronecker lifting, we design a computationally efficient estimator for which we provide provable finite-sample guarantees. Together, these results provide a coherent probabilistic framework and practical algorithms for learning from tensor-valued data.
Paper Structure (35 sections, 29 theorems, 215 equations, 3 figures, 1 table, 1 algorithm)

This paper contains 35 sections, 29 theorems, 215 equations, 3 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Tensor spaces and notation
  3. The inner product space of tensors
  4. Linear maps between tensor spaces
  5. Two spectral norms
  6. Computations with tensors
  7. Self-adjoint mappings
  8. Gaussian tensors
  9. Tensor probabilistic PCA
  10. Model setup
  11. Special cases and related models
  12. Matrix/tensor–normal distributions.
  13. Deterministic Tucker–rank models.
  14. CP-rank factor models.
  15. Bayesian Tucker and CP literature—relation to TPCA.
  16. ...and 20 more sections

Key Result

Lemma 2.4

For Tucker maps $\|\mathsf{F}_{\boldsymbol A}\|_{\rm ml}= \|\mathsf{F}_{\boldsymbol A}\|=\prod_{k=1}^r\|A_k\|$.

Figures (3)

  • Figure 1: Under \ref{['eq:assnoneptyint']}, we have $f_1(\tau)\le \tau$ precisely for $\tau\in[\tau_0,\tau_1]$ (blue). Moreover, if $\tau$ lies in the green region $(0,\tau_0)$, then $f_1(\tau)$ also lies in $(0,\tau_0)$.
  • Figure 2: Case 2 (power iteration): $\mathrm{Err}$ (left) and $\widehat{\sigma^2}$ (right) versus iteration $L$, with $95\%$ CIs over 50 replications. The error metric is the Procrustes-aligned $\sqrt{n_k m_k}$-normalized error defined at the start of the section.
  • Figure 3: Case 3 (power iteration): averaged $\mathrm{Err}$ for $\widehat{A}_1,\widehat{A}_2,\widehat{A}_3$ and the relative error of $\widehat{\sigma^2}$ across $(\sigma^2,N)$, with $95\%$ CIs over 50 replications. The error metric is the Procrustes-aligned $\sqrt{n_k m_k}$-normalized error defined at the start of the section.

Theorems & Definitions (62)

  • Remark 2.1
  • Definition 2.2: Tucker maps
  • Remark 2.3
  • Lemma 2.4
  • Lemma 2.5
  • Proposition 2.6
  • Definition 2.7
  • Lemma 2.8
  • Lemma 2.9
  • Remark 2.10
  • ...and 52 more