Wireless Distributed Matrix-Vector Multiplication using Over-the-Air Computation and Analog Coding

TL;DR

This paper studies distributed matrix–vector multiplication in DML over wireless channels using over-the-air computation to exploit signal superposition and avoid increasing bandwidth. It introduces column-wise partitioning of ${\mathbf A}$ and a two-phase OTA protocol to adapt task granularity to worker capabilities and mitigate stragglers. It analyzes the resulting mean squared error under fading channels and proposes an analog coding scheme, with comparisons to digital coded schemes (CM). Simulation results show OTA-based methods can match CM performance while using fewer radio resources, and analog coding further improves NMSE, highlighting practical potential for large-scale wireless DML.

Abstract

In this paper, we propose an over-the-air (OTA)-based approach for distributed matrix-vector multiplications in the context of distributed machine learning (DML). Thanks to OTA computation, the column-wise partitioning of a large matrix enables efficient workload distribution among workers (i.e., local computing nodes) based on their computing capabilities. In addition, without requiring additional bandwidth, it allows the system to remain scalable even as the number of workers increases to mitigate the impact of slow workers, known as stragglers. However, despite the improvements, there are still instances where some workers experience deep fading and become stragglers, preventing them from transmitting their results. By analyzing the mean squared error (MSE), we demonstrate that incorporating more workers in the OTA-based approach leads to MSE reduction without the need for additional radio resources. Furthermore, we introduce an analog coding scheme to further enhance the performance and compare it with conventional coded multiplication (CM) schemes. Through simulations, it is shown that the OTA-based approach achieves comparable performance to CM schemes while potentially requiring fewer radio resources.

Figures (8)

• Figure 1: An example of two-phase communication between a platform and multiple workers for distributed machine learning.
• Figure 2: Outage probabilities for the 3 different cases.
• Figure 3: Normalized individual MSE with the upper-bound in \ref{['EQ:bMSE1']}.
• Figure 4: An example of block-wise encoding and decoding. Note that shaded blocks represent coded data matrices, $\tilde{{\bf A}}_{k,l}$.
• Figure 5: NMSE of the OTA-based approach as a function of the maximum transmit power when $K = L = 10$ and $M_k = Q_l = 10$.