Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximately Optimal Core Shapes for Tensor Decompositions

Mehrdad Ghadiri, Matthew Fahrbach, Gang Fu, Vahab Mirrokni

TL;DR

This paper addresses the problem of selecting an optimal core shape for size-constrained Tucker tensor decompositions by formulating the Tucker packing problem. It establishes NP-hardness and delivers a polynomial-time approximation scheme (PTAS) by reducing a surrogate loss based on higher-order singular values to a 2D knapsack problem with a partition matroid, plus budget-splitting techniques and grid-search refinements. The authors extend the framework to tree tensor networks and provide an IP-based implementation (HOSVD-IP) that is competitive with, and often faster than, greedy baselines that use the true reconstruction loss, achieving substantial speedups. Empirical results on several real-world tensors demonstrate effective core-shape selection and substantial run-time advantages, enabling scalable core-shape optimization for large-scale tensors.

Abstract

This work studies the combinatorial optimization problem of finding an optimal core tensor shape, also called multilinear rank, for a size-constrained Tucker decomposition. We give an algorithm with provable approximation guarantees for its reconstruction error via connections to higher-order singular values. Specifically, we introduce a novel Tucker packing problem, which we prove is NP-hard, and give a polynomial-time approximation scheme based on a reduction to the 2-dimensional knapsack problem with a matroid constraint. We also generalize our techniques to tree tensor network decompositions. We implement our algorithm using an integer programming solver, and show that its solution quality is competitive with (and sometimes better than) the greedy algorithm that uses the true Tucker decomposition loss at each step, while also running up to 1000x faster.

Approximately Optimal Core Shapes for Tensor Decompositions

TL;DR

This paper addresses the problem of selecting an optimal core shape for size-constrained Tucker tensor decompositions by formulating the Tucker packing problem. It establishes NP-hardness and delivers a polynomial-time approximation scheme (PTAS) by reducing a surrogate loss based on higher-order singular values to a 2D knapsack problem with a partition matroid, plus budget-splitting techniques and grid-search refinements. The authors extend the framework to tree tensor networks and provide an IP-based implementation (HOSVD-IP) that is competitive with, and often faster than, greedy baselines that use the true reconstruction loss, achieving substantial speedups. Empirical results on several real-world tensors demonstrate effective core-shape selection and substantial run-time advantages, enabling scalable core-shape optimization for large-scale tensors.

Abstract

This work studies the combinatorial optimization problem of finding an optimal core tensor shape, also called multilinear rank, for a size-constrained Tucker decomposition. We give an algorithm with provable approximation guarantees for its reconstruction error via connections to higher-order singular values. Specifically, we introduce a novel Tucker packing problem, which we prove is NP-hard, and give a polynomial-time approximation scheme based on a reduction to the 2-dimensional knapsack problem with a matroid constraint. We also generalize our techniques to tree tensor network decompositions. We implement our algorithm using an integer programming solver, and show that its solution quality is competitive with (and sometimes better than) the greedy algorithm that uses the true Tucker decomposition loss at each step, while also running up to 1000x faster.
Paper Structure (42 sections, 21 theorems, 78 equations, 4 figures, 1 table, 2 algorithms)

This paper contains 42 sections, 21 theorems, 78 equations, 4 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Our contributions and techniques
  3. Related works
  4. Core shape constraints.
  5. Low-rank tensor decomposition.
  6. Preliminaries
  7. Notation.
  8. Tensor products.
  9. Tucker decomposition.
  10. Reduction to HOSVD Tucker packing
  11. Higher-order singular value decomposition
  12. Tucker packing problem
  13. Algorithm
  14. Warm-up: Connections to multiple-choice knapsack
  15. Prefix sums transformation.
  16. ...and 27 more sections

Key Result

Theorem 3.1

Any tensor $\bm{\mathscr{X}}\in\mathbb{R}^{I_1\times\cdots\times I_N}$ can be written as where each $\mathbf{U}^{(n)} \in {\mathbb{R}}^{I_n \times I_n}$ is an orthogonal matrix and $\bm{\mathscr{S}} \in \mathbb{R}^{I_1\times\cdots\times I_N}$ is a tensor with subtensors $\bm{\mathscr{S}}_{i_n = \alpha}$, obtained by fixing the $n$-th index to $\alpha$, that have the properties: Furthermore, the

Figures (4)

  • Figure 1: Pareto frontier of core shapes $\mathbf{r} \in [20]^3$ for hyperspectral tensor $\bm{\mathscr{X}} \in \mathbb{R}^{1024 \times 1344 \times 33}$. Plots the RRE, i.e., $L(\bm{\mathscr{X}},\mathbf{r}) / \lVert\bm{\mathscr{X}}\rVert_{\textnormal{F}}^2$, as a function of compression rate. RRE-greedy builds core shapes by computing Tucker decompositions at each step. HOSVD-IP is \ref{['alg:total-size-Tucker']} with integer programming, which builds core shapes via a surrogate packing problem on higher-order singular values.
  • Figure 2: Comparison of five core shape solvers on four real-world tensors (columns) for increasing values of the Tucker decomposition size budget $c \le 100,000$. The plots in the top row are the HOSVD Tucker packing objective value $f(\mathbf{r})$ for the core shape solutions $\mathbf{r}$, the middle row is the RRE, and the bottom row is the running time of each algorithm in seconds.
  • Figure 3: Tree tensor network corresponding to a Tucker decomposition.
  • Figure 4: Tree tensor network example corresponding to a hierarchical Tucker decomposition.

Theorems & Definitions (42)

  • Theorem 3.1: HOSVD
  • Theorem 3.2: HOSVD; hackbusch2019tensor
  • Definition 3.3
  • Lemma 3.3
  • Definition 3.4: Tucker packing problem
  • Theorem 3.5
  • Lemma 3.5
  • Remark 3.6
  • Theorem 4.1: lawler1977fast
  • Definition 4.2
  • ...and 32 more