Multivariate Polynomial Codes for Efficient Matrix Chain Multiplication in Distributed Systems

TL;DR

This work tackles straggler-induced latency in distributed matrix chain multiplication by moving beyond univariate coded computing. It introduces two multivariate polynomial coding schemes, MV1 and MV2, that substantially reduce storage and upload costs at the expense of higher per-worker computation. MV1 uses separate univariate encodings per matrix, while MV2 uses a shared-variable bivariate encoding to further reduce data traffic, with MV2 providing the strongest practical balance for long chains. Numerical results show that multivariate schemes, especially MV2, achieve bounded storage overhead independent of chain length, offering scalable solutions for large-scale distributed linear algebra tasks.$M$ and associated blocks are discussed within a finite field framework, and decoding considerations are addressed for multivariate codes.

Abstract

We study the problem of computing matrix chain multiplications in a distributed computing cluster. In such systems, performance is often limited by the straggler problem, where the slowest worker dominates the overall computation latency. To resolve this issue, several coded computing strategies have been proposed, primarily focusing on the simplest case: the multiplication of two matrices. These approaches successfully alleviate the straggler effect, but they do so at the expense of higher computational complexity and increased storage needs at the workers. However, in many real-world applications, computations naturally involve long chains of matrix multiplications rather than just a single two-matrix product. Extending univariate polynomial coding to this setting has been shown to amplify the costs -- both computation and storage overheads grow significantly, limiting scalability. In this work, we propose two novel multivariate polynomial coding schemes specifically designed for matrix chain multiplication in distributed environments. Our results show that while multivariate codes introduce additional computational cost at the workers, they can dramatically reduce storage overhead compared to univariate extensions. This reveals a fundamental trade-off between computation and storage efficiency, and highlights the potential of multivariate codes as a practical solution for large-scale distributed linear algebra tasks.

Figures (4)

• Figure 1: Distributed computational system.
• Figure 2: Computation overhead, $\delta$%, as a function of the number of partitions.
• Figure 3: Storage overhead, $\delta_{S}$%, as a function of the number of partitions.
• Figure 4: Storage overhead, $\delta_{S}$%, as a function of the number of workers.