When Bayesian Tensor Completion Meets Multioutput Gaussian Processes: Functional Universality and Rank Learning

TL;DR

This work proposes a rank-revealing functional Bayesian tensor completion (RR-FBTC) method and establishes the universal approximation property of the model for continuous multi-dimensional signals, demonstrating its expressive power in a concise format.

Abstract

Functional tensor decomposition can analyze multi-dimensional data with real-valued indices, paving the path for applications in machine learning and signal processing. A limitation of existing approaches is the assumption that the tensor rank-a critical parameter governing model complexity-is known. However, determining the optimal rank is a non-deterministic polynomial-time hard (NP-hard) task and there is a limited understanding regarding the expressive power of functional low-rank tensor models for continuous signals. We propose a rank-revealing functional Bayesian tensor completion (RR-FBTC) method. Modeling the latent functions through carefully designed multioutput Gaussian processes, RR-FBTC handles tensors with real-valued indices while enabling automatic tensor rank determination during the inference process. We establish the universal approximation property of the model for continuous multi-dimensional signals, demonstrating its expressive power in a concise format. To learn this model, we employ the variational inference framework and derive an efficient algorithm with closed-form updates. Experiments on both synthetic and real-world datasets demonstrate the effectiveness and superiority of the RR-FBTC over state-of-the-art approaches. The code is available at https://github.com/OceanSTARLab/RR-FBTC.

Figures (12)

• Figure 1: The allocation process in two- and three-dimensional scenarios, where $i_1$, $i_2$, and $i_3$ represent the axes of the modes (e.g., longitude, latitude, and depth). The observed data, indicated by red points, are allocated as entries of a discrete tensor $\hbox{\boldmath $\mathcal{Y}$}$ (the gray box), in which the dimension of the $k$-th mode is $N_k=|\mathcal{S}_k|$, where $\mathcal{S}_k=\{i_k^1,\cdots,i_k^{N_k}\}$ is the real-valued coordinate set containing the unique values of mode-$k$'s indices from all the data.
• Figure 2: The rank-revealing functional prior. The tensor entry $x_{\mathbf{i}}$, the brown point on the left side of the equal sign, is decomposed using the CP format \ref{['eq:7']}, which is a sum of the products of latent functions $u_r^k(i_k)$. The black curves in red circles represent these latent functions $\{u^k_r(\cdot)\}$, which are modeled via MOGP priors \ref{['eq: mogp']}. The green lines depict discrete vectors that contain the sampled function values at coordinate sets $\{\mathcal{S}_k\}$. The vector $\boldsymbol{\gamma}$ controls the powers of the processes across different modes, enabling rank-revealing functional modeling. An element of $\boldsymbol{\gamma}^{-1}$ near 0 (denoted by a blank box) results in the pruning of the corresponding rank-1 component during iteration. In contrast, a positive $\boldsymbol{\gamma}^{-1}$ signifies active components in the decomposition.
• Figure 3: [Synthetic discrete data] Tensor rank estimates and RRSEs of the proposed RR-FBTC and FBCP under different SNRs. The blue dashed line is the ground-truth rank and the error bars show the standard deviation.
• Figure 4: [Synthetic discrete data] RRSE of LRTFR with different rank settings. The blue dashed line is the RRSE of RR-FBTC.
• Figure 5: [Synthetic continuous data] The estimated latent factors and respective uncertainty of RR-FBTC for the synthetic continuous data. The SR = 20% and SNR = 10 dB.