An Efficient and Robust Projection Enhanced Interpolation Based Tensor Train Decomposition

TL;DR

The paper tackles the challenge of achieving accurate, data-sparse representations for high-dimensional tensors by enhancing skeletonized interpolation-based TT decompositions. It extends projection-based robustness to the TT setting via Projection Enhanced Interpolation-Based Decomposition (PEID), combining oversampling of unselected pivots with oblique projections to improve accuracy without incurring prohibitive computational costs. The authors develop dimension-parallel and sequential TT-PEID algorithms, along with two-sided variants and rounding, and demonstrate substantial accuracy gains across synthetic Hilbert tensors, kernel-based tensors, and Maxwellian distributions, while maintaining scalable complexity. The approach integrates with existing TT frameworks (e.g., TTACA) and tools like TnTorch, offering a practical, robust path for high-dimensional tensor compression with interpretable skeletons and improved stability in noisy or degenerative settings.

Abstract

The tensor-train (TT) format is a data-sparse tensor representation commonly used in high dimensional data approximations. In order to represent data with interpretability in data science, researchers develop data-centric skeletonized low rank approximations. However, these methods might still suffer from accuracy degeneracy, nonrobustness, and high computation costs. In this paper, given existing skeletonized TT approximations, we propose a family of projection enhanced interpolation based algorithms to further improve approximation accuracy while keeping low computational complexity. We do this as a postprocessing step to existing interpolative decompositions, via oversampling data not in skeletons to include more information and selecting subsets of pivots for faster projections. We illustrate the performances of our proposed methods with extensive numerical experiments. These include up to 10D synthetic datasets such as tensors generated from kernel functions, and tensors constructed from Maxwellian distribution functions that arise in kinetic theory. Our results demonstrate significant accuracy improvement over original skeletonized TT approximations, while using limited amount of computational resources.

Figures (13)

• Figure 1: The TT format with TT core size $\pmb{s} = (s_0,\ldots,s_d)$. Each entry of a tensor is represented by the product of $d$ matrices, where the $k$th matrix in the "train" is selected based on the value of $i_k$.
• Figure 2: Various of ways to build CUR approximations of a $100\times 100$ Hilbert matrix. Left: Demonstrating the stability of using $C$ and $R$ for the construction of $U = C^\dagger AR^\dagger$ (labeled "Direct"), versus using $Q_C$ and $Q_R$ for $U = Q_C^TAQ_R$ (labeled "QR"). Right: Skeletonized approximation errors when using varied $U$ construction. Using \ref{['eq:matrix_HIP_ON']} is termed "P-Full", \ref{['eq:matrix_HIP_oblique_sp']} is termed "OP-ACA", \ref{['eq:matrix_HIP_oblique']} where $\mathcal{S},\mathcal{T}$ come from applying QDEIM to $Q_C$ and $Q_R$ is termed "OP-QDEIM", and a random subset of 50 indices for $\mathcal{S},\mathcal{T}$ is termed "OP-Random".
• Figure 3: Diagram of the working pieces behind the matrix level approximation using oversampled columns and rows, and submatrix selection for oblique projections.
• Figure 4: Visual representation of the nestedness of the index selection for the oversampling sets $\mathcal{K}_{\leq j}$ in \ref{['alg:TT_update_parallel']}. $\mathcal{K}_{\leq 1}$ can be freely selected, but the candidate region for $\mathcal{K}_{\leq 2}$ is the region corresponding to all locations in $X_2$ where the first index value is a member of $\mathcal{K}_{\leq 1}$, i.e. blocks of rows in $X_2$. This behavior continues for $\mathcal{K}_{\leq 3}$ and futher.
• Figure 5: Construction of $F_{\leq j}$ inside of \ref{['alg:TT_update_sequential']}. The left image corresponds to the locations $\mathcal{I}_{\leq j}\cup \mathcal{K}_{\leq j}$ in the $j$-th unfolding $X_j$. For each index in $\mathcal{I}_{\leq j}\cup \mathcal{K}_{\leq j}$, we separate it out into $j$ indices. Then, we perform a sequence of vector matrix products, where we use the $j$ indices to take the vector from the first core $\mathcal{T}_1$, and the remaining matrices from the already computed cores $\mathcal{T}_2,\dots,\mathcal{T}_{j}$. This multiplication forms one row of $F_{\leq j}$.