Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing Tensor Contraction Paths: A Greedy Algorithm Approach With Improved Cost Functions

Sheela Orgler, Mark Blacher

TL;DR

The paper tackles the problem of efficiently finding tensor contraction paths, which critically affect computational cost in applications like model counting and quantum circuits. It builds on the greedy framework of opt_einsum and introduces a modified algorithm that evaluates multiple cost functions at runtime to adapt to the problem instance, aiming to minimize intermediates or FLOPs. Through extensive experiments across domains, the approach demonstrates faster path computation and the ability to solve larger problems than competing methods, with cotengra excelling in some graph-related tasks. The method offers a practical, scalable alternative for contraction path optimization and includes open-source code to facilitate adoption and further research.

Abstract

Finding efficient tensor contraction paths is essential for a wide range of problems, including model counting, quantum circuits, graph problems, and language models. There exist several approaches to find efficient paths, such as the greedy and random greedy algorithm by Optimized Einsum (opt_einsum), and the greedy algorithm and hypergraph partitioning approach employed in cotengra. However, these algorithms require a lot of computational time and resources to find efficient contraction paths. In this paper, we introduce a novel approach based on the greedy algorithm by opt_einsum that computes efficient contraction paths in less time. Moreover, with our approach, we are even able to compute paths for large problems where modern algorithms fail.

Optimizing Tensor Contraction Paths: A Greedy Algorithm Approach With Improved Cost Functions

TL;DR

The paper tackles the problem of efficiently finding tensor contraction paths, which critically affect computational cost in applications like model counting and quantum circuits. It builds on the greedy framework of opt_einsum and introduces a modified algorithm that evaluates multiple cost functions at runtime to adapt to the problem instance, aiming to minimize intermediates or FLOPs. Through extensive experiments across domains, the approach demonstrates faster path computation and the ability to solve larger problems than competing methods, with cotengra excelling in some graph-related tasks. The method offers a practical, scalable alternative for contraction path optimization and includes open-source code to facilitate adoption and further research.

Abstract

Finding efficient tensor contraction paths is essential for a wide range of problems, including model counting, quantum circuits, graph problems, and language models. There exist several approaches to find efficient paths, such as the greedy and random greedy algorithm by Optimized Einsum (opt_einsum), and the greedy algorithm and hypergraph partitioning approach employed in cotengra. However, these algorithms require a lot of computational time and resources to find efficient contraction paths. In this paper, we introduce a novel approach based on the greedy algorithm by opt_einsum that computes efficient contraction paths in less time. Moreover, with our approach, we are even able to compute paths for large problems where modern algorithms fail.
Paper Structure (16 sections, 6 equations, 6 figures, 2 tables)

This paper contains 16 sections, 6 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Tensors
  4. Einstein Summation Notation
  5. Tensor Hypernetworks
  6. Tensor Contraction
  7. Related Work
  8. The Algorithm
  9. The Standard Greedy Algorithm
  10. The Modified Greedy Algorithm
  11. Modifications.
  12. The Cost Functions.
  13. Experiments
  14. Setup
  15. Results
  16. ...and 1 more sections

Figures (6)

  • Figure 1: Graphical representation of a matrix, a tensor, and a tensor network.
  • Figure 2: Example of a tensor hypernetwork.
  • Figure 3: Two example contraction trees for computing the result of a tensor network with four tensors. On the left is a balanced contraction tree, and on the right is a skewed one.
  • Figure 4: Results for the computation of 128 paths.
  • Figure 5: Time for the computation of 128 paths.
  • ...and 1 more figures