Tensor Generalized Approximate Message Passing

TL;DR

This work tackles low-rank tensor inference for completion and decomposition by introducing TeG-AMP, an AMP-based algorithm tailored to the Tensor Ring (TR) decomposition. The method derives a belief-propagation–inspired approximation in the high-dimensional limit, enabling tractable updates that exploit tensor structure and generalize across TR, CP, TT, and Tucker forms. A CP-based simplification, TeS-AMP, is also presented to reduce complexity for CP-rank tensors. Experimental results on synthetic TR tensors and MNIST data demonstrate that TeG-AMP achieves significantly better recovery than AltMin and BiG-AMP by leveraging the full tensor structure, albeit with higher computational cost, which motivates the damping strategies and TeS-AMP as a practical alternative.

Abstract

We propose a tensor generalized approximate message passing (TeG-AMP) algorithm for low-rank tensor inference, which can be used to solve tensor completion and decomposition problems. We derive TeG-AMP algorithm as an approximation of the sum-product belief propagation algorithm in high dimensions where the central limit theorem and Taylor series approximations are applicable. As TeG-AMP is developed based on a general TR decomposition model, it can be directly applied to many low-rank tensor types. Moreover, our TeG-AMP can be simplified based on the CP decomposition model and a tensor simplified AMP is proposed for low CP-rank tensor inference problems. Experimental results demonstrate that the proposed methods significantly improve recovery performances since it takes full advantage of tensor structures.

Figures (21)

• Figure 1: An illustration of the factor-graph for tensor generalized inference problem based on TR decomposition. The factor nodes $\{ p({v_{\mathbf{x}} | u_{\mathbf{x}}}) \}$ are represented as 'boxes', and the number of factor nodes is $\prod_{i}{N_i}$. The variable nodes $\{ \mathcal{Z}_i (:,x_i,:)\}$ are represented as 'circles', and the number of variable nodes is $\sum_{i}{N_i}$. Each factor node $p\left({ v_{\mathbf{n}} | u_{\mathbf{n}}}\right)$ is connected to $d$ variable nodes $\{ \mathcal{Z}_{i}(:,x_i,:):i=1,...,d\}$, and each variable node $\mathcal{Z}_{i}(:,x_i,:)$ is connected to $\prod_{i'=1}^{d}{N_{i'}}/N_{i}$ factor nodes $\{p\left({ v_{\mathbf{x}'} | u_{\mathbf{x}'}}\right):x'_i= x_i\}$.
• Figure 2: Comparison results for TR-rank tensor $\mathcal{U} \in \mathbb{R}^{6 \times 7 \times 8}$. The sampling rate in noisy cases is $100\%$. Upper-Left: TR-rank $2,3,3$ in noiseless cases; Upper-Right: TR-rank $2,2,2$ in noiseless cases; Below-Left: TR-rank $2,3,3$ in noisy cases; Below-Right: TR-rank $2,2,2$ in noisy cases.
• Figure 3: MNIST digits with size $28\times 28 \times 6$ and TR rank $14\times 14 \times 6$. The sampling rate is $40\%$. Line 1: Ground truth; Line 2: Sampling Results; Line 3: Recovered digits via AltMin; Line 4: Recovered digits via BiG-AMP; Line 5: Recovered digits via TeG-AMP.
• Figure 4: Factor graph of postulated posterior $p(x|y)$.
• Figure 5: A graphical representation of the TR decomposition.

Theorems & Definitions (13)

• Remark 3.1: Differences
• Remark 3.2: Structure
• Remark 3.3: Structure
• Definition A.1
• Remark D.1
• Remark D.2
• Remark E.1
• Remark E.2
• Remark E.3
• Remark H.1