Scalable Bayesian Tensor Ring Factorization for Multiway Data Analysis
Zerui Tao, Toshihisa Tanaka, Qibin Zhao
TL;DR
This work tackles the bottlenecks of scalability and discrete-data handling in Bayesian Tensor Ring factorization. It introduces a scalable SBTR framework that uses a weighted TR with a nonparametric Multiplicative Gamma Process prior, along with Polya-Gamma augmentation for binary data, a Gibbs sampler for posterior inference, and an online EM algorithm for large-scale tensors. The approach enables automatic rank adaptation and efficient inference, achieving strong rank-estimation and completion performance on both continuous and binary multiway data, with improved scalability over prior VI-based methods. Overall, SBTR advances practical Bayesian tensor analysis for large, heterogeneous datasets, enabling uncertainty-aware modeling and robust factorization in real-world applications.
Abstract
Tensor decompositions play a crucial role in numerous applications related to multi-way data analysis. By employing a Bayesian framework with sparsity-inducing priors, Bayesian Tensor Ring (BTR) factorization offers probabilistic estimates and an effective approach for automatically adapting the tensor ring rank during the learning process. However, previous BTR method employs an Automatic Relevance Determination (ARD) prior, which can lead to sub-optimal solutions. Besides, it solely focuses on continuous data, whereas many applications involve discrete data. More importantly, it relies on the Coordinate-Ascent Variational Inference (CAVI) algorithm, which is inadequate for handling large tensors with extensive observations. These limitations greatly limit its application scales and scopes, making it suitable only for small-scale problems, such as image/video completion. To address these issues, we propose a novel BTR model that incorporates a nonparametric Multiplicative Gamma Process (MGP) prior, known for its superior accuracy in identifying latent structures. To handle discrete data, we introduce the Pólya-Gamma augmentation for closed-form updates. Furthermore, we develop an efficient Gibbs sampler for consistent posterior simulation, which reduces the computational complexity of previous VI algorithm by two orders, and an online EM algorithm that is scalable to extremely large tensors. To showcase the advantages of our model, we conduct extensive experiments on both simulation data and real-world applications.