Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HaTT: Hadamard avoiding TT recompression

Zhonghao Sun, Jizu Huang, Chuanfu Xiao, Chao Yang

TL;DR

This work tackles the bottleneck caused by rank growth in Hadamard products of tensor train (TT) tensors. It introduces HaTT, a Hadamard product-free recompression framework that uses right-to-left partial contractions and PKP-aware sketching to avoid forming the explicit Hadamard product, significantly reducing both computational and memory costs. HaTT yields two variants, HaTT-1 and HaTT-2, and demonstrates substantial speedups over traditional TT-Rounding and randomized methods across Fourier-series Hadamard products, random TT products, power iteration, and nonlinear PDE applications (notably Allen–Cahn), while maintaining high accuracy. The approach also offers theoretical insight into numerical rank propagation and presents practical benefits for large-scale TT computations, with potential extensions to quantum TT tensors and tolerance-controlled recompression. Overall, HaTT provides a scalable, efficient tool for TT-based nonlinear operations that frequently arise in scientific computing and data analysis.

Abstract

The Hadamard product of tensor train (TT) tensors is a fundamental nonlinear operation in scientific computing and data analysis. However, due to its tendency to significantly increase TT ranks, the Hadamard product poses a major computational challenge in TT tensor-based algorithms. To address this, it is crucial to develop recompression algorithms that mitigate the effects of this rank increase. Existing recompression algorithms require an explicit representation of the Hadamard product, resulting in high computational and storage costs. In this work, we propose a Hadamard avoiding TT recompression (HaTT) algorithm, which reduces both computational complexity and storage requirements. By leveraging the structure of the Hadamard product in TT tensors and exploiting its Hadamard product-free property, the HaTT algorithm achieves significantly lower complexity compared to existing TT recompression methods. This is confirmed through both complexity analysis and numerical experiments. Furthermore, the HaTT algorithm is applied to solve the Allen--Cahn equation, achieving substantial speedup over existing TT recompression algorithms without sacrificing accuracy.

HaTT: Hadamard avoiding TT recompression

TL;DR

This work tackles the bottleneck caused by rank growth in Hadamard products of tensor train (TT) tensors. It introduces HaTT, a Hadamard product-free recompression framework that uses right-to-left partial contractions and PKP-aware sketching to avoid forming the explicit Hadamard product, significantly reducing both computational and memory costs. HaTT yields two variants, HaTT-1 and HaTT-2, and demonstrates substantial speedups over traditional TT-Rounding and randomized methods across Fourier-series Hadamard products, random TT products, power iteration, and nonlinear PDE applications (notably Allen–Cahn), while maintaining high accuracy. The approach also offers theoretical insight into numerical rank propagation and presents practical benefits for large-scale TT computations, with potential extensions to quantum TT tensors and tolerance-controlled recompression. Overall, HaTT provides a scalable, efficient tool for TT-based nonlinear operations that frequently arise in scientific computing and data analysis.

Abstract

The Hadamard product of tensor train (TT) tensors is a fundamental nonlinear operation in scientific computing and data analysis. However, due to its tendency to significantly increase TT ranks, the Hadamard product poses a major computational challenge in TT tensor-based algorithms. To address this, it is crucial to develop recompression algorithms that mitigate the effects of this rank increase. Existing recompression algorithms require an explicit representation of the Hadamard product, resulting in high computational and storage costs. In this work, we propose a Hadamard avoiding TT recompression (HaTT) algorithm, which reduces both computational complexity and storage requirements. By leveraging the structure of the Hadamard product in TT tensors and exploiting its Hadamard product-free property, the HaTT algorithm achieves significantly lower complexity compared to existing TT recompression methods. This is confirmed through both complexity analysis and numerical experiments. Furthermore, the HaTT algorithm is applied to solve the Allen--Cahn equation, achieving substantial speedup over existing TT recompression algorithms without sacrificing accuracy.
Paper Structure (16 sections, 4 theorems, 55 equations, 13 figures, 3 tables, 4 algorithms)

This paper contains 16 sections, 4 theorems, 55 equations, 13 figures, 3 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Tensor notations and operations
  4. Tensor train tensor and its Hadamard product
  5. Recompression of tensor train tensor
  6. Hadamard avoiding TT recompression
  7. Right-to-left partial contraction for Hadamard product
  8. HaTT
  9. Complexity analysis
  10. Numerical experiments
  11. Example 1: Hadamard product of Fourier series functions
  12. Comparison between HaTT-1 and HaTT-2
  13. Example 2: Hadamard product of random TT tensors with different TT ranks
  14. Example 3: application of HaTT in power iteration
  15. Example 4: application of HaTT in solving nonlinear PDEs
  16. ...and 1 more sections

Key Result

Theorem 3.1

Let $\bm{\mathcal{A}} \in \mathbb{R}^{n_{1} \times n_{2} \times \cdots \times n_{d}}$ be a TT tensor with separation ranks $\{r^{\text{sep}}_{0}, r^{\text{sep}}_{1}, \dots, r^{\text{sep}}_{d}\}$. Suppose that we generate a set of sketch matrices $\{\bm{W}^{(k)}\}_{k = 1}^{d - 1}$ of $\bm{\mathcal{A}

Figures (13)

  • Figure 1: Tensor network diagram for a TT tensor $\bm{\mathcal{A}}$.
  • Figure 1: The process of right-to-left partial contraction
  • Figure 1: (a) Relative errors, (b) running times, and (c) speedups for Example 1.
  • Figure 2: The process of HPCRL for direct representation \ref{['HPCRL--1']}.
  • Figure 2: Singular values of $\mathcal{V}\left(\bm{\mathcal{T}}_{\bm{\mathcal{A}}, 1:k}\right)$, $\mathcal{H}\left(\bm{\mathcal{T}}_{\bm{\mathcal{A}}, k + 1:d}\right)$, $\bm{W}^{(k-1)}$, and $\bm{A}_{\langle k - 1 \rangle}$ with $k = 4, 5$.
  • ...and 8 more figures

Theorems & Definitions (11)

  • Definition 2.1: Random TT tensor
  • Theorem 3.1
  • Proof 1
  • Definition 3.2: Absolute quasi-numerical ranks
  • Definition 3.3: Relative quasi-numerical ranks
  • Remark 3.4
  • Lemma 3.5
  • Proof 2
  • Lemma 3.6: Theorem 3.3.16 in horn1994topics
  • Theorem 3.7
  • ...and 1 more