Fundamental Limits of Multi-User Distributed Computing of Linearly Separable Functions

TL;DR

This work analyzes the fundamental limits of multi-user distributed computing of linearly separable functions, formalizing a setting with $L$ users, $N$ servers, a per-server computation budget $M$, and a connectivity constraint $\Delta$. It introduces a left-nullspace–based achievability scheme that jointly designs task assignment and server transmissions, achieving a rate $R_{\text{ach}}(K,L,M,\Delta)=\Delta\left\lceil\frac{L}{\Delta}\right\rceil\left\lceil\frac{K}{\Delta+M-1}\right\rceil$, and derives matrix-factorization–based converses to characterize performance over finite and real fields. The finite-field converse shows that, as $q\to\infty$, the rate cannot be smaller than $LK/(\Delta+M-1)$, and under divisibility conditions ($\Delta\mid L$, $(\Delta+M-1)\mid K$) the proposed scheme is optimal up to constant factors (3 when $q\ge K$); similarly, a DoF-based bound over $\mathbb{R}$ yields near-optimality within a factor of 4 under the same divisibility constraints. The results substantially tighten the understanding of the communication–computation tradeoff in multi-user distributed computing with linearly separable tasks and provide practical guidance for task assignment and coded transmission design.

Abstract

This work establishes the fundamental limits of the classical problem of multi-user distributed computing of linearly separable functions. In particular, we consider a distributed computing setting involving $L$ users, each requesting a linearly separable function over $K$ basis subfunctions from a master node, who is assisted by $N$ distributed servers. At the core of this problem lies a fundamental tradeoff between communication and computation: each server can compute up to $M$ subfunctions, and each server can communicate linear combinations of their locally computed subfunctions outputs to at most $Δ$ users. The objective is to design a distributed computing scheme that reduces the communication cost (total amount of data from servers to users), and towards this, for any given $K$, $L$, $M$, and $Δ$, we propose a distributed computing scheme that jointly designs the task assignment and transmissions, and shows that the scheme achieves optimal performance in the real field under various conditions using a novel converse. We also characterize the performance of the scheme in the finite field using another converse based on counting arguments.

Figures (1)

• Figure 1: A multi-user distributed computing system.

Theorems & Definitions (14)

• Theorem 1
• proof
• Example 1
• Remark 1
• Theorem 2
• proof
• Corollary 1
• proof
• Remark 2
• Theorem 3: Optimality gap