Straggler-Aware Coded Polynomial Aggregation

TL;DR

This work extends coded polynomial aggregation (CPA) to straggler-aware distributed systems by imposing a pre-specified non-straggler pattern. It develops a CPA framework where exact recovery of the weighted polynomial aggregation $Y=\sum_{k=0}^{K-1} w_k\,F(\mathbf{X}_k)$ is achieved through interpolation-based encoding, decoding, and orthogonality constraints, with feasibility governed by the intersection structure of the pattern. The authors establish necessary and sufficient conditions for feasibility, derive a mathematically grounded intersection-size threshold $I^*$ that guarantees feasibility, and provide an explicit construction method for the CPA scheme when $I \ge I^*$. Simulations corroborate the theory, showing a sharp feasibility transition at $I^*$ and delivering practical guidance for deploying straggler-aware CPA in distributed computing environments.

Abstract

Coded polynomial aggregation (CPA) in distributed computing systems enables the master to directly recover a weighted aggregation of polynomial computations without individually decoding each term, thereby reducing the number of required worker responses. However, existing CPA schemes are restricted to an idealized setting in which the system cannot tolerate stragglers. In this paper, we extend CPA to straggler-aware distributed computing systems with a pre-specified non-straggler pattern, where exact recovery is required for a given collection of admissible non-straggler sets. Our main results show that exact recovery of the desired aggregation is achievable with fewer worker responses than that required by polynomial codes based on individual decoding, and that feasibility is characterized by the intersection structure of the non-straggler patterns. In particular, we establish necessary and sufficient conditions for exact recovery in straggler-aware CPA. We identify an intersection-size threshold that is sufficient to guarantee exact recovery. When the number of admissible non-straggler sets is sufficiently large, we further show that this threshold is necessary in a generic sense. We also provide an explicit construction of feasible CPA schemes whenever the intersection size exceeds the derived threshold. Finally, simulations verify our theoretical results by demonstrating a sharp feasibility transition at the predicted intersection threshold.

Figures (1)

• Figure 1: Empirical feasibility $p_{eq}(I)$ versus the intersection size $I$ for $K=5$, $S=2$ and $G_{\text{max}} = 7$. The vertical gray dashed line indicates the sufficient threshold $I^*$ in Theorem \ref{['th-bound-case1']}. For $d=1$, we consider $N=6,5,4$, corresponding to maximum intersection sizes $I=4,3,2$, respectively; when $I \le 2$, the curves for $N=5$ and $N=4$ overlap. For $d=2$, we consider $N=10,9,8,7$, corresponding to maximum intersection sizes $I=8,7,6,5$, respectively; when $I \le 5$, the curves for $N=9,8,7$ overlap.

Theorems & Definitions (9)

• Definition 1: Non-Straggler Pattern
• Definition 2: Feasibility over a Non-Straggler Pattern
• Definition 3: CPA Based on Individual Decoding
• Lemma 1
• Theorem 1
• Remark 1
• Lemma 2
• Theorem 2
• Corollary 1