Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large-Scale Bayesian Tensor Reconstruction: An Approximate Message Passing Solution

Bingyang Cheng, Zhongtao Chen, Yichen Jin, Hao Zhang, Chen Zhang, Edmud Y. Lam, Yik-Chung Wu

TL;DR

The paper tackles scalable Bayesian CPD for large, potentially incomplete tensors with unknown CP-rank and noise power. It introduces CP-GAMP, a GAMP-based approach derived from loopy belief propagation, employing CLT and Taylor approximations to avoid expensive matrix inversions. Bernoulli-Gaussian priors are used to enable automatic CP-rank learning, and an EM routine jointly estimates the rank and the noise power during inference. Empirical results demonstrate substantial runtime reductions compared with VI-based methods and TC-AMP, while preserving reconstruction quality, including a synthetic 100×100×100 tensor with rank-20 and 80% missing data and image inpainting tasks.

Abstract

Tensor CANDECOMP/PARAFAC decomposition (CPD) is a fundamental model for tensor reconstruction. Although the Bayesian framework allows for principled uncertainty quantification and automatic hyperparameter learning, existing methods do not scale well for large tensors because of high-dimensional matrix inversions. To this end, we introduce CP-GAMP, a scalable Bayesian CPD algorithm. This algorithm leverages generalized approximate message passing (GAMP) to avoid matrix inversions and incorporates an expectation-maximization routine to jointly infer the tensor rank and noise power. Through multiple experiments, for synthetic 100x100x100 rank 20 tensors with only 20% elements observed, the proposed algorithm reduces runtime by 82.7% compared to the state-of-the-art variational Bayesian CPD method, while maintaining comparable reconstruction accuracy.

Large-Scale Bayesian Tensor Reconstruction: An Approximate Message Passing Solution

TL;DR

The paper tackles scalable Bayesian CPD for large, potentially incomplete tensors with unknown CP-rank and noise power. It introduces CP-GAMP, a GAMP-based approach derived from loopy belief propagation, employing CLT and Taylor approximations to avoid expensive matrix inversions. Bernoulli-Gaussian priors are used to enable automatic CP-rank learning, and an EM routine jointly estimates the rank and the noise power during inference. Empirical results demonstrate substantial runtime reductions compared with VI-based methods and TC-AMP, while preserving reconstruction quality, including a synthetic 100×100×100 tensor with rank-20 and 80% missing data and image inpainting tasks.

Abstract

Tensor CANDECOMP/PARAFAC decomposition (CPD) is a fundamental model for tensor reconstruction. Although the Bayesian framework allows for principled uncertainty quantification and automatic hyperparameter learning, existing methods do not scale well for large tensors because of high-dimensional matrix inversions. To this end, we introduce CP-GAMP, a scalable Bayesian CPD algorithm. This algorithm leverages generalized approximate message passing (GAMP) to avoid matrix inversions and incorporates an expectation-maximization routine to jointly infer the tensor rank and noise power. Through multiple experiments, for synthetic 100x100x100 rank 20 tensors with only 20% elements observed, the proposed algorithm reduces runtime by 82.7% compared to the state-of-the-art variational Bayesian CPD method, while maintaining comparable reconstruction accuracy.

Paper Structure

This paper contains 35 sections, 51 equations, 5 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background and Probabilistic Model
  3. Tensor CPD
  4. Probabilistic Model for Tensor CPD
  5. Method
  6. Factor Graph
  7. Loopy Belief Propagation
  8. From Sum-Product to CP-GAMP
  9. Sum-Product Algorithm
  10. Approximation of Factor-to-Variable Messages
  11. Approximation of Variable-to-Factor Messages
  12. Uniform Messages Emitted from the Identical Variable Node
  13. Special Form of CP-GAMP via Prior and Likelihood Incorporation
  14. Hyperparameter Learning
  15. CP-Rank Learning
  16. ...and 20 more sections

Figures (5)

  • Figure 1: The cumulative distribution function of $0.5\mathcal{N}(x;0,1)+0.5\delta(x)$ and $\mathcal{N}(x;0,1)$.
  • Figure 2: The factor graph for the tensor CPD model of toy-sized problems with dimensions $I_1 = 3, I_2=3, I_3=3, \text{and}\, N=3$.
  • Figure 3: Simulation results on $3$-order tensor reconstruction under SNR of 10 dB. The upper row shows the performance results, and the lower row shows the runtime results. Left: different dimensions with CP-rank of 20 and observation ratio of 0.2; Middle: different CP-rank with tensor dimension of $100\times 100\times 100$ and observation ratio of 0.2; Right: different observation ratios with tensor dimension of $100\times 100 \times 100$ and CP-rank of 20.
  • Figure 4: Experimental result on image inpainting under SNR of 10 dB and observation ratio of $30\%$. NMSE and runtime are shown in the subtitles of the sub-figures.
  • Figure 5: The visualization results of image inpainting with observation ratio of 30% and SNR of 10 dB