Non-Linearly Separable Distributed Computing: A Sparse Tensor Factorization Approach

TL;DR

An achievable scheme is designed, employing novel fixed-support SVD-based tensor factorization methods and careful multi-dimensional tiling of subtensors, yielding computation and communication protocols whose costs are derived here, and which are shown to perform substantially better than the state of art.

Abstract

The work considers the $N$-server distributed computing setting with $K$ users requesting functions that are arbitrary multi-variable polynomial evaluations of $L$ real (potentially non-linear) basis subfunctions. Our aim is to seek efficient task-allocation and data-communication techniques that reduce computation and communication costs. Towards this, we take a tensor-theoretic approach, in which we represent the requested non-linearly decomposable functions using a properly designed tensor $\bar{\mathcal{F}}$, whose sparse decomposition into a tensor $\bar{\mathcal{E}}$ and matrix $\mathbf{D}$ directly defines the task assignment, connectivity, and communication patterns. We here design an achievable scheme, employing novel fixed-support SVD-based tensor factorization methods and careful multi-dimensional tiling of subtensors, yielding computation and communication protocols whose costs are derived here, and which are shown to perform substantially better than the state of art.

Figures (4)

• Figure 1: The lossless $(K,N,L,\Gamma,\Delta, \{P_{\ell},\Lambda_{\ell}\}_{\ell\in [L]})$ distributed computing setting with a coordinator node, $N$ servers, and $K$ users.
• Figure 2: The figure on the left illustrates the support constraints $\mathbf{I}$ and $\bar{\mathcal{J}}$ on $\mathbf{D}$ and $\bar{\mathcal{E}}$ respectively. The constraints $\mathbf{I}(:,1)$ and $\bar{\mathcal{J}}(1,:,:)$ on the columns and rows of $\mathbf{D}$ and $\bar{\mathcal{E}}$ respectively are colored green, $\mathbf{I}(:,2)$ and $\bar{\mathcal{J}}(2,:,:)$ are colored cyan and $\mathbf{I}(:,3)$ and $\bar{\mathcal{J}}(3,:,:)$ are colored red. The product of a column with a row of the same color yields the corresponding rank-one contribution support $\bar{\mathcal{S}}_{n}(\mathbf{I},\bar{\mathcal{J}}),n=1,2,3,$ as described in Definition \ref{['def-r1']}, and as illustrated on the right side of the figure.
• Figure 3: This figure illustrates three different rank-one contribution supports $\bar{\mathcal{S}}_{1},\bar{\mathcal{S}}_{2},\bar{\mathcal{S}}_{3}$, where the first two fall into the same equivalence class $\bar{\mathcal{S}}_{\mathcal{P}_1} = \bar{\mathcal{S}}_{1} = \bar{\mathcal{S}}_{2}$, while $\bar{\mathcal{S}}_{\mathcal{P}_2} = \bar{\mathcal{S}}_{3}$.
• Figure 4: Corresponding to Example \ref{['single-shot-example-simple']}, this figure illustrates the partitioning of $\mathcal{\bar{F}}$ into $8$ tiles of size $(2\times 2\times 2)$, and the sparse tiling of $\mathbf{D}$ and $\mathcal{\bar{E}}$ with tiles $\mathbf{L}_j$ and $\bar{\mathcal{R}}_j$, respectively, resulting in the full tiling of $\bar{\mathcal{F}} = \bar{\mathcal{E}} \times_{1} \mathbf{D}$.

Theorems & Definitions (18)

• Example 1
• Theorem 1
• proof
• Definition 1
• Lemma 1
• proof
• Definition 2
• Definition 3
• Definition 4
• Definition 5: le2023spurious