Fast and Accurate CP-HIFI Tensor Decompositions: Exploiting Kronecker Structure
Johannes J. Brust, Tamara G. Kolda
Abstract
Tensor decompositions are a fundamental tool in scientific computing and data analysis. In many applications -- such as simulation data on irregular grids, surrogate modeling for parameterized PDEs, or spectroscopic measurements -- the data has both discrete and continuous structure, and may only be observed at scattered sample points. The CP-HIFI (hybrid infinite-finite) decomposition generalizes the Canonical Polyadic (CP) tensor decomposition to settings where some factors are finite-dimensional vectors and others are functions drawn from infinite-dimensional spaces. The decomposition can be applied to a fully observed tensor (aligned) or, when only scattered observations are available, to a sparsely sampled tensor (unaligned). Current methods compute CP-HIFI factors by solving a sequence of dense linear systems arising from regularized least-squares problems to fit reproducing Kernel Hilbert space (RKHS) representations to the data, but these direct solves become computationally prohibitive as problem size grows. We propose new algorithms that achieve the same accuracy while being orders of magnitude faster. For aligned tensors, we exploit the Kronecker structure of the system to efficiently compute its eigendecomposition without ever forming the full system, reducing the solve to independent scalar equations. For unaligned tensors, we introduce a preconditioned conjugate gradient method, exploiting the problem's structure for fast matrix-vector products and efficient preconditioning. In our experiments, the proposed methods speed up the solution up to 500x compared to the prior naive direct methods, in line with the reduction in the theoretical computational complexity.