Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximating Tensor Network Contraction with Sketches

Mike Heddes, Igor Nunes, Tony Givargis, Alex Nicolau

TL;DR

This work presents the first method capable of approximating arbitrary tensor network contractions, including those of cyclic tensor networks, and shows that the existing sketching methods require a computational complexity that grows exponentially with the number of contractions.

Abstract

Tensor network contraction is a fundamental mathematical operation that generalizes the dot product and matrix multiplication. It finds applications in numerous domains, such as database systems, graph theory, machine learning, probability theory, and quantum mechanics. Tensor network contractions are computationally expensive, in general requiring exponential time and space. Sketching methods include a number of dimensionality reduction techniques that are widely used in the design of approximation algorithms. The existing sketching methods for tensor network contraction, however, only support acyclic tensor networks. We present the first method capable of approximating arbitrary tensor network contractions, including those of cyclic tensor networks. Additionally, we show that the existing sketching methods require a computational complexity that grows exponentially with the number of contractions. We present a second method, for acyclic tensor networks, whose space and time complexity depends only polynomially on the number of contractions.

Approximating Tensor Network Contraction with Sketches

TL;DR

This work presents the first method capable of approximating arbitrary tensor network contractions, including those of cyclic tensor networks, and shows that the existing sketching methods require a computational complexity that grows exponentially with the number of contractions.

Abstract

Tensor network contraction is a fundamental mathematical operation that generalizes the dot product and matrix multiplication. It finds applications in numerous domains, such as database systems, graph theory, machine learning, probability theory, and quantum mechanics. Tensor network contractions are computationally expensive, in general requiring exponential time and space. Sketching methods include a number of dimensionality reduction techniques that are widely used in the design of approximation algorithms. The existing sketching methods for tensor network contraction, however, only support acyclic tensor networks. We present the first method capable of approximating arbitrary tensor network contractions, including those of cyclic tensor networks. Additionally, we show that the existing sketching methods require a computational complexity that grows exponentially with the number of contractions. We present a second method, for acyclic tensor networks, whose space and time complexity depends only polynomially on the number of contractions.
Paper Structure (27 sections, 14 theorems, 37 equations, 4 figures, 1 table, 5 algorithms)

This paper contains 27 sections, 14 theorems, 37 equations, 4 figures, 1 table, 5 algorithms.

Table of Contents

  1. Introduction
  2. Technical overview
  3. Preliminaries
  4. Count sketch
  5. Tensor sketch
  6. Recursive sketch
  7. Related work
  8. Methods
  9. Without loss of generality assumptions
  10. Approximating any tensor network contraction
  11. Exponential lower bound
  12. Improved approximation for acyclic tensor network contraction
  13. Main result
  14. Applications of tensor network contraction
  15. Join size estimation
  16. ...and 12 more sections

Key Result

Theorem 1

For every $p \in \mathbb{N}^{+}$, any order-$q_k$ tensors ${\mathcal{X}}_k$ for $k \in \lbrack p\rbrack$, and every $\epsilon, \delta > 0$, there exists an $(\epsilon, \delta)$-ATNC of a full TNC that can be computed in time $O((pm \log m + q N) \log 1/\delta)$ using $O(m p \log 1/\delta)$ space, wi

Figures (4)

  • Figure 1: Tensor diagram of an example tensor network
  • Figure 2: Tensor diagram of a simple acyclic (left) and cyclic (right) tensor network
  • Figure 3: Tensor diagram of an example acyclic tensor network
  • Figure 4: Example SQL query (left) and corresponding tensor network diagram (right). For all $k \in \lbrack4\rbrack$, every component ${\mathcal{X}}_k({\mathbf{i}})$ denotes the frequency of ${\mathbf{i}}$ in relation $R_k$.

Theorems & Definitions (43)

  • Definition 1: Tensor contraction
  • Definition 2: Tensor product
  • Definition 3: Tensor network contraction (TNC)
  • Example 1
  • Definition 4: Approximate tensor network contraction (ATNC)
  • Theorem 1
  • Definition 5: Hadamard product
  • Definition 6: Kronecker product
  • Definition 7: Row-wise Kronecker product
  • Definition 8: Circular convolution
  • ...and 33 more