Projected Tensor-Tensor Products for Efficient Computation of Optimal Multiway Data Representations

TL;DR

This work addresses the high computational and storage costs of matrix-mimetic tensor algebras by introducing a projected tensor-tensor product based on a unitary tall-and-skinny matrix \\mathbf{Q}. The authors establish matrix mimeticity and Eckart-Young optimality for the projected product \\star_{\\mathbf{Q}^H}^\\ extprime and develop a compressible variant, \\star_{\\mathbf{Q}^H}^\\ extprime-SVDII, which offers favorable approximation quality at reduced storage. They prove that the projected approach can outperform the traditional \\star_{\\mathbf{M}}-SVD and even the truncated HOSVD in many settings, and they validate the theory with extensive video and hyperspectral data experiments. The results indicate substantial savings in storage and computation while maintaining high approximation quality, providing a scalable path for large-scale multiway data representations. The work also releases open-source code to promote reproducibility and further development.

Abstract

Tensor decompositions have become essential tools for feature extraction and compression of multiway data. Recent advances in tensor operators have enabled desirable properties of standard matrix algebra to be retained for multilinear factorizations. Behind this matrix-mimetic tensor operation is an invertible matrix whose size depends quadratically on certain dimensions of the data. As a result, for large-scale multiway data, the invertible matrix can be computationally demanding to apply and invert and can lead to inefficient tensor representations in terms of construction and storage costs. In this work, we propose a new projected tensor-tensor product that relaxes the invertibility restriction to reduce computational overhead and still preserves fundamental linear algebraic properties. The transformation behind the projected product is a tall-and-skinny matrix with unitary columns, which depends only linearly on certain dimensions of the data, thereby reducing computational complexity by an order of magnitude. We provide extensive theory to prove the matrix mimeticity and the optimality of compressed representations within the projected product framework. We further prove that projected-product-based approximations outperform a comparable, non-matrix-mimetic tensor factorization. We support the theoretical findings and demonstrate the practical benefits of projected products through numerical experiments on video and hyperspectral imaging data.

Figures (12)

• Figure 1: Illustration of key partitions of third-order tensors.
• Figure 2: Comparison of $\star_{\mathbf{M}}$-pipeline (top) and $\star_{\mathbf{Q}^H}^\prime$-pipeline (bottom) for multiplying tensors. Above each operation, we describe the computational cost for dense numerical linear algebra operations with an easy-to-invert matrix $\mathbf{M}$ (see Halko2011). In this setting, the cost of the $\star_{\mathbf{M}}$-product depends quadratically on $n_3$, i.e., $\mathcal{O}(n_3^2)$, whereas the $\star_{\mathbf{Q}^H}^\prime$-product has only a linear dependence, i.e., $\mathcal{O}(n_3)$. We note that inverting a general matrix $\mathbf{M}$ could increase the $\star_{\mathbf{M}}$-product reverse transform cost by a factor of $n_3$. Conversely, if $\mathbf{M}$ could be implemented via a fast transformation (e.g., fft), then the cost of the of transforms could decrease to $\mathcal{O}(n_3\log n_3)$.
• Figure 3: Illustration of $\boldsymbol{\mathcal{A}} \times_3 \mathbf{U}_3^H$ where $\mathbf{U}_3$ is the left-singular matrix from the mode-$3$ unfolding of $\boldsymbol{\mathcal{A}}$; that is, $\mathbf{A}_{(3)} = \mathbf{U}_3 \text{\boldmath$\mathbf{\Sigma}$}_3 \mathbf{V}_3^H$. The frontal slices of $\boldsymbol{\mathcal{A}} \times_3 \mathbf{U}_3^H$ have Frobenius norm equal to the singular values of the mode-$3$ unfolding. From the ordering of the singular values (indicated by the various shades of magenta), the relative importance of each transformed frontal slice decays from front (dark magenta) to back (light magenta).
• Figure 4: Empirical results for traffic video compression.
• Figure 5: Empirical results for shuttle video compression.

Theorems & Definitions (24)

• Definition 2.1: mode-$3$ unfolding/folding
• Definition 2.2: mode-$3$ product
• Definition 2.3: facewise product
• Definition 2.4: $\star_{\mathbf{M}}$-product
• Definition 3.1: $\star_{\mathbf{Q}^H}^\prime$-product
• Definition 3.2: $\star_{\mathbf{Q}^H}^\prime$-identity tensor
• Definition 3.3: $\star_{\mathbf{Q}^H}^\prime$-conjugate transpose
• Definition 3.4: $\star_{\mathbf{Q}^H}^\prime$-unitary
• Definition 3.5: facewise diagonal (f-diagonal)
• proof