Strassen's algorithm is not optimally accurate

TL;DR

The paper tackles the numerical stability of sub-cubic matrix multiplication algorithms, focusing on Strassen-type methods, by formulating a unified forward-error bound for recursive bilinear operators via Hadamard-based HM representations and a growth factor $\gamma$. It shows that optimizing $\gamma$ along Strassen's isotropy orbit in the Frobenius norm, together with heuristics to minimize operation counts and a basis-sparsification approach, yields practical variants with improved accuracy while preserving the $n^{\log_2 7}$ exponent. The authors establish a concrete best-possible growth-factor value on a restricted orbit (approximately $12.066$ for their optimal point) and a theoretical lower bound around $11.755$ that cannot be achieved by any 7-multiplication HM representation, highlighting inherent limitations. They provide open-source tools (PLinOpt and mFMM) and demonstrate experimentally that their best variants outperform Strassen and Winograd in numerical accuracy, while maintaining favorable leading-term time complexity due to basis sparsification. Overall, the work offers a principled framework to derive more accurate fast matrix multiplication algorithms using tensor isotropy and basis transformations, with practical impact on high-performance linear algebra.

Abstract

We propose a non-commutative algorithm for multiplying 2x2 matrices using 7 coefficient products. This algorithm reaches simultaneously a better accuracy in practice compared to previously known such fast algorithms, and a time complexity bound with the best currently known leading term (obtained via alternate basis sparsification). To build this algorithm, we consider matrix and tensor norms bounds governing the stability and accuracy of numerical matrix multiplication. First, we reduce those bounds by minimizing a growth factor along the unique orbit of Strassen's 2x2-matrix multiplication tensor decomposition. Second, we develop heuristics for minimizing the number of operations required to realize a given bilinear formula, while further improving its accuracy. Third, we perform an alternate basis sparsification that improves on the time complexity constant and mostly preserves the overall accuracy.