Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Structured Matrix Multiplication

Derya Malak

TL;DR

This work addresses reducing the communication needed for distributed computation of matrix products over finite-field sources. It introduces nonlinear source mappings followed by a structured Korner-Marton encoding to compute inner products and matrix products, and provides achievable sum-rate regions that can be strictly smaller than unstructured Slepian-Wolf coding in certain regimes, with potential unbounded gains for specific source correlations. The results extend from inner products to symmetric and general square matrix products, including vector-wise embeddings and hybrid KM-OR schemes, and are supported by corollaries and numerical comparisons. Overall, the approach offers significant compression gains for distributed linear-algebra operations, with implications for coded computation and secure distributed matrix multiplication.

Abstract

We devise achievable encoding schemes for distributed source compression for computing inner products, symmetric matrix products, and more generally, square matrix products, which are a class of nonlinear transformations. To that end, our approach relies on devising nonlinear mappings of distributed sources, which are then followed by the structured linear encoding scheme, introduced by Körner and Marton. For different computation scenarios, we contrast our findings on the achievable sum rate with the state of the art to demonstrate the possible savings in compression rate. When the sources have special correlation structures, it is possible to achieve unbounded gains, as demonstrated by the analysis and numerical simulations.

Distributed Structured Matrix Multiplication

TL;DR

This work addresses reducing the communication needed for distributed computation of matrix products over finite-field sources. It introduces nonlinear source mappings followed by a structured Korner-Marton encoding to compute inner products and matrix products, and provides achievable sum-rate regions that can be strictly smaller than unstructured Slepian-Wolf coding in certain regimes, with potential unbounded gains for specific source correlations. The results extend from inner products to symmetric and general square matrix products, including vector-wise embeddings and hybrid KM-OR schemes, and are supported by corollaries and numerical comparisons. Overall, the approach offers significant compression gains for distributed linear-algebra operations, with implications for coded computation and secure distributed matrix multiplication.

Abstract

We devise achievable encoding schemes for distributed source compression for computing inner products, symmetric matrix products, and more generally, square matrix products, which are a class of nonlinear transformations. To that end, our approach relies on devising nonlinear mappings of distributed sources, which are then followed by the structured linear encoding scheme, introduced by Körner and Marton. For different computation scenarios, we contrast our findings on the achievable sum rate with the state of the art to demonstrate the possible savings in compression rate. When the sources have special correlation structures, it is possible to achieve unbounded gains, as demonstrated by the analysis and numerical simulations.
Paper Structure (12 sections, 7 theorems, 41 equations, 3 figures)

This paper contains 12 sections, 7 theorems, 41 equations, 3 figures.

Table of Contents

  1. Introduction
  2. System Model and Main Results
  3. Distributed Computation of Inner Products of Sources
  4. Distributed Computation of Symmetric Matrices
  5. Distributed Computation of Square Matrices
  6. Proof of Proposition \ref{['prop:KW_sum_rate_for_inner_product']}
  7. Proof of Corollary \ref{['cor:innerproduct_length_m_binary']}
  8. Proof of Proposition \ref{['prop:KW_sum_rate_for_inner_product_linear_source_mappings']}
  9. Proof of Proposition \ref{['prop:KW_sum_rate_for_symmetric_matrix_product']}
  10. Proof of Proposition \ref{['prop:KW_sum_rate_for_inner_product+vs_SW_sum_rate']}
  11. Proof of Proposition \ref{['prop:KW_sum_rate_for_general_matrix_product']}
  12. Proof of Corollary \ref{['cor:general_matrix_q3']}

Key Result

Proposition 1

(Distributed inner product computation.) Given two sequences of random vectors ${\bf A}=^{\intercal}\in\mathbb{F}_q^{m\times 1}$ and ${\bf B}=^{\intercal}\in\mathbb{F}_q^{m\times 1}$ of even length $m$, generated by two correlated memoryless $q$-ary sources, where ${\bf A}_1,{\bf A}_2,{\bf B}_1,{\bf where ${\bf U}, {\bf V}\in \mathbb{F}_q^{m/2\times 1}$ are vector variables, and $W\in \mathbb{F}_q

Figures (3)

  • Figure 1: Gain, $\eta$ (cf. (\ref{['gain_DSBS_channel_KM']}) in Corollary \ref{['cor:innerproduct_length_m_binary']}. The flat (yellow) line marks $\eta=1$.
  • Figure 2: Rate comparisons for various source PMFs. (Left) Corollary \ref{['cor:innerproduct_length_m_binary']} for $m=2$. (Middle) $m=1$, and $a,b$ are DSBSs. (Right) $m=2$, and $\{a_i,b_i\}_{i=1}^2$ are DSBSs.
  • Figure 3: Rate (in log scale) versus $p$ for distributed computing of (Left) symmetric matrices ${\bf A}^{\intercal}{\bf B}={\bf B}^{\intercal}{\bf A}$ via distributed multiplication of matrices ${\bf A}, {\bf B}\in \mathbb{F}_2^{m\times m}$ for different $m$, and (Right) square matrices via distributed matrix multiplication for different $m$ and $l=2$, where the joint source PMF is given in Corollary \ref{['cor:general_matrix_q3']}.

Theorems & Definitions (14)

  • Proposition 1
  • proof
  • Corollary 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3
  • proof
  • Proposition 4
  • proof
  • ...and 4 more