SamBaTen: Sampling-based Batch Incremental Tensor Decomposition

TL;DR

SamBaTen tackles the challenge of incrementally updating CP tensor decompositions for dynamically growing tensors by operating on compact sampled summaries of the data. The method samples incoming updates, runs parallel CP decompositions on these samples, and projects the results back to update the global factor matrices, with rank-aware quality control to handle rank-deficient updates. Empirical results show SamBaTen attaining competitive accuracy while delivering 25–30x faster performance and enabling Scale to tensors up to 100K per dimension, where many baselines fail. This approach offers a practical, scalable solution for real-time analysis of evolving multimodal data in domains like social networks and large-scale knowledge bases.

Abstract

Tensor decompositions are invaluable tools in analyzing multimodal datasets. In many real-world scenarios, such datasets are far from being static, to the contrary they tend to grow over time. For instance, in an online social network setting, as we observe new interactions over time, our dataset gets updated in its "time" mode. How can we maintain a valid and accurate tensor decomposition of such a dynamically evolving multimodal dataset, without having to re-compute the entire decomposition after every single update? In this paper we introduce SaMbaTen, a Sampling-based Batch Incremental Tensor Decomposition algorithm, which incrementally maintains the decomposition given new updates to the tensor dataset. SaMbaTen is able to scale to datasets that the state-of-the-art in incremental tensor decomposition is unable to operate on, due to its ability to effectively summarize the existing tensor and the incoming updates, and perform all computations in the reduced summary space. We extensively evaluate SaMbaTen using synthetic and real datasets. Indicatively, SaMbaTen achieves comparable accuracy to state-of-the-art incremental and non-incremental techniques, while being 25-30 times faster. Furthermore, SaMbaTen scales to very large sparse and dense dynamically evolving tensors of dimensions up to 100K x 100K x 100K where state-of-the-art incremental approaches were not able to operate.

Figures (11)

• Figure 1: SamBaTen vs state-of-art techniques: Our proposed method SamBaTen outperforms state-of-the-art baselines while maintaining competitive accuracy.
• Figure 2: Slices of 3-order tensor (a) horizontal $\underline{ \mathbf{X} }(i,:,:)$ (b) lateral $\underline{ \mathbf{X} }(:,j,:)$, (c) frontal $\underline{ \mathbf{X} }(:,j,:)$.
• Figure 3: Real life dynamic data examples (a) collecting new location update from satellite to GPS recommendations systems (b) growing Facebook interaction between people over the time.
• Figure 4: SamBaTen: Sampling-based Batch Incremental Tensor Decomposition: 1) Sample incoming tensor into sub-tensors, 2) run parallel decompositions on the samples, 3) project back the results into the original space, and, finally, 4) update the incrementally growing factor matrix $\mathbf{C}$.
• Figure 5: Experimental results for CPU time (sec) for (a) dense tensor (b) sparse tensor

Theorems & Definitions (2)

• Lemma 1
• proof