Multi-Server Multi-Function Distributed Computation

TL;DR

This work addresses the communication bottleneck in multi-server multi-function distributed computation where a user requests multiple non-linear functions over distributed datasets. It adopts Körner's characteristic-graph entropy to jointly capture data statistics, correlations, and function structure, deriving a general upper bound on the sum-rate that can be achieved when any $N_r$ servers cooperate in decoding. The main theoretical contributions include a general bound (Theorem 1) and specialized results for correlated Boolean functions and multi-linear functions under cyclic data placement, demonstrating significant gains over prior linear-coding baselines, especially with skewed data in a binary field. The framework offers principled compression limits for distributed functional computation and shows practical impact in reducing communication for complex tasks in cloud and edge computing settings, paving the way for efficient, scalable coded computing under general function classes.

Abstract

The work here studies the communication cost for a multi-server multi-task distributed computation framework, and does so for a broad class of functions and data statistics. Considering the framework where a user seeks the computation of multiple complex (conceivably non-linear) tasks from a set of distributed servers, we establish communication cost upper bounds for a variety of data statistics, function classes and data placements across the servers. To do so, we proceed to apply, for the first time here, Körner's characteristic graph approach -- which is known to capture the structural properties of data and functions -- to the promising framework of multi-server multi-task distributed computing. Going beyond the general expressions, and in order to offer clearer insight, we also consider the well-known scenario of cyclic dataset placement and linearly separable functions over the binary field, in which case our approach exhibits considerable gains over the state of art. Similar gains are identified for the case of multi-linear functions.

Figures (6)

• Figure 1: The gain $\eta_{lin}$ of the characteristic graph approach for $K_c=1$, in Subsection \ref{['ex:binary_lin_sep']} (Scenario I). (Left) $\rho=0$ for various distributed topologies. (Right) The correlation model given as (\ref{['ex:corr_model_probability']}) for $\mathcal{T}(30, 30, 1, 11, 20)$ with different $\epsilon$ values.
• Figure 2: Colorings of graphs in Subsection \ref{['ex:binary_lin_sep']} (Scenario II). (Top Left-Right) Characteristic graphs $G_{X_1}$ and $G_{X_2}$, respectively. (Bottom Left-Right) The minimum conditional entropy colorings of $G_{X_1}$ given $c_{G_{X_2}}$, and $G_{X_2}$ given $c_{G_{X_1}}$, respectively.
• Figure 3: $\eta_{lin}$ in (\ref{['eta_scenario_II_rho_0']}) versus $\epsilon$, for distributed computing of $f_1=W_2$ and $f_2=W_2+ W_3$, where $K_c=2$, $N_r=2$, with $\rho=0$, in Subsection \ref{['ex:binary_lin_sep']} (Scenario II).
• Figure 4: $\eta_{lin}$ versus $\epsilon$, for distributed computing of $f_1=W_2$ and $f_2=W_2+ W_3$, where $K_c=2$, $N_r=2$, in Subsection \ref{['ex:binary_lin_sep']}, using different joint PMF models for $P_{W_2,W_3}$ (Scenario II). (Left) $\eta_{lin}$ in (\ref{['eta_scenario_II_rho_nonzero_TableII']}) for the joint PMF in Table \ref{['tab:joint_pmf']}, for different values of $p$. (Right) $\eta_{lin}$ for the joint PMF in (\ref{['ex:corr_model_probability']}), for different values of $\rho$ .
• Figure 5: $\eta_{lin}$ in a logarithmic scale versus $\epsilon$ for $K_c$ demanded functions for various values of $K_c$, with $\rho=0$ for different topologies, as detailed in Subsection \ref{['ex:binary_lin_sep']} (Scenario III).

Theorems & Definitions (12)

• Theorem 1
• proof
• Proposition 1
• proof
• Proposition 2
• proof
• Proposition 3
• proof
• Definition 1
• Example 1