Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploiting the Structure in Tensor Decompositions for Matrix Multiplication

Manuel Kauers, Jakob Moosbauer, Isaac Wood

TL;DR

The paper tackles the problem of reducing the exponent of matrix multiplication by exploiting structure in tensor decompositions. It introduces a recursive, structure-aware algorithm that shares inputs/outputs across recursive calls and leverages tensor powers and cyclic permutations to gain exponent savings without reducing the base multiplication count. For the notable $6x6$ case, it lowers the exponent from $2.8075$ to $2.8016$, and shows the approach can yield further improvements with carefully chosen decompositions, albeit with a higher leading coefficient. The authors also provide a complexity analysis, decomposition searches, and simulations indicating potential practical benefits for very large matrix sizes, highlighting a path toward more practical structured algorithms within the generalized ASI framework.

Abstract

We present a new algorithm for fast matrix multiplication using tensor decompositions which have special features. Thanks to these features we obtain exponents lower than what the rank of the tensor decomposition suggests. In particular for $6\times 6$ matrix multiplication we reduce the exponent of the recent algorithm by Moosbauer and Poole from $2.8075$ to $2.8016$, while retaining a reasonable leading coefficient.

Exploiting the Structure in Tensor Decompositions for Matrix Multiplication

TL;DR

The paper tackles the problem of reducing the exponent of matrix multiplication by exploiting structure in tensor decompositions. It introduces a recursive, structure-aware algorithm that shares inputs/outputs across recursive calls and leverages tensor powers and cyclic permutations to gain exponent savings without reducing the base multiplication count. For the notable case, it lowers the exponent from to , and shows the approach can yield further improvements with carefully chosen decompositions, albeit with a higher leading coefficient. The authors also provide a complexity analysis, decomposition searches, and simulations indicating potential practical benefits for very large matrix sizes, highlighting a path toward more practical structured algorithms within the generalized ASI framework.

Abstract

We present a new algorithm for fast matrix multiplication using tensor decompositions which have special features. Thanks to these features we obtain exponents lower than what the rank of the tensor decomposition suggests. In particular for matrix multiplication we reduce the exponent of the recent algorithm by Moosbauer and Poole from to , while retaining a reasonable leading coefficient.
Paper Structure (6 sections, 7 theorems, 46 equations, 1 figure, 1 table, 1 algorithm)

This paper contains 6 sections, 7 theorems, 46 equations, 1 figure, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. divide less, conquer more
  4. Complexity Analysis
  5. Specific Decompositions
  6. Simulation

Key Result

Theorem 3

If $\langle n,m,p \rangle\leq \langle r \rangle$ then there is an algorithm to multiply $N\times N$ matrices in $\operatorname{O}(N^{3\log_{nmp}(r)})$ operations in $R$.

Figures (1)

  • Figure 1: Simulated operation count for our algorithm and Strassen's algorithm

Theorems & Definitions (13)

  • Definition 1
  • Definition 2
  • Theorem 3
  • Theorem 4
  • proof
  • Theorem 5: Asymptotic Sum Inequality (ASI)
  • Corollary 6
  • Theorem 7
  • proof
  • Theorem 8
  • ...and 3 more