Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncoded Storage Coded Transmission Elastic Computing with Straggler Tolerance in Heterogeneous Systems

Xi Zhong, Joerg Kliewer, Mingyue Ji

TL;DR

USCTEC tackles elastic computing in heterogeneous systems by combining uncoded storage with coded transmission using $L$-ary Lagange codes to tolerate stragglers while performing matrix-matrix multiplication. It provides optimal USCTEC schemes without storage constraints for a fixed speed realization, and a heuristic algorithm for general speed distributions under storage constraints, via the subproblems $(l,\boldsymbol{s},\boldsymbol{\sigma})$-LP and $(\boldsymbol{\theta},\rho)$-DP. The framework yields a two-stage design: partitioning $\boldsymbol{A}$ into $G$ blocks, distributing storage $\boldsymbol{e}$, and constructing Lagrange-code-based transmissions to enable decoding with any $L$ out of $L+S$ evaluators. Compared with cyclic storage baselines, USCTEC achieves reduced storage size and comparable or improved expected computation time in heterogeneous, straggler-prone settings, with illustrative scenarios and performance tradeoffs.

Abstract

In 2018, Yang et al. introduced a novel and effective approach, using maximum distance separable (MDS) codes, to mitigate the impact of elasticity in cloud computing systems. This approach is referred to as coded elastic computing. Some limitations of this approach include that it assumes all virtual machines have the same computing speeds and storage capacities, and it cannot tolerate stragglers for matrix-matrix multiplications. In order to resolve these limitations, in this paper, we introduce a new combinatorial optimization framework, named uncoded storage coded transmission elastic computing (USCTEC), for heterogeneous speeds and storage constraints, aiming to minimize the expected computation time for matrix-matrix multiplications, under the consideration of straggler tolerance. Within this framework, we propose optimal solutions with straggler tolerance under relaxed storage constraints. Moreover, we propose a heuristic algorithm that considers the heterogeneous storage constraints. Our results demonstrate that the proposed algorithm outperforms baseline solutions utilizing cyclic storage placements, in terms of both expected computation time and storage size.

Uncoded Storage Coded Transmission Elastic Computing with Straggler Tolerance in Heterogeneous Systems

TL;DR

USCTEC tackles elastic computing in heterogeneous systems by combining uncoded storage with coded transmission using -ary Lagange codes to tolerate stragglers while performing matrix-matrix multiplication. It provides optimal USCTEC schemes without storage constraints for a fixed speed realization, and a heuristic algorithm for general speed distributions under storage constraints, via the subproblems -LP and -DP. The framework yields a two-stage design: partitioning into blocks, distributing storage , and constructing Lagrange-code-based transmissions to enable decoding with any out of evaluators. Compared with cyclic storage baselines, USCTEC achieves reduced storage size and comparable or improved expected computation time in heterogeneous, straggler-prone settings, with illustrative scenarios and performance tradeoffs.

Abstract

In 2018, Yang et al. introduced a novel and effective approach, using maximum distance separable (MDS) codes, to mitigate the impact of elasticity in cloud computing systems. This approach is referred to as coded elastic computing. Some limitations of this approach include that it assumes all virtual machines have the same computing speeds and storage capacities, and it cannot tolerate stragglers for matrix-matrix multiplications. In order to resolve these limitations, in this paper, we introduce a new combinatorial optimization framework, named uncoded storage coded transmission elastic computing (USCTEC), for heterogeneous speeds and storage constraints, aiming to minimize the expected computation time for matrix-matrix multiplications, under the consideration of straggler tolerance. Within this framework, we propose optimal solutions with straggler tolerance under relaxed storage constraints. Moreover, we propose a heuristic algorithm that considers the heterogeneous storage constraints. Our results demonstrate that the proposed algorithm outperforms baseline solutions utilizing cyclic storage placements, in terms of both expected computation time and storage size.
Paper Structure (13 sections, 3 theorems, 17 equations, 1 figure, 1 table, 2 algorithms)

This paper contains 13 sections, 3 theorems, 17 equations, 1 figure, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. Storage Placement and Storage Selections
  4. Communication Phase
  5. Computing Phase and Decoding Phase
  6. USCTEC with Straggler Tolerance Problem Formulation
  7. Optimal USCTEC Schemes without Storage Constraints for A Given Speed Realization
  8. Problem Analysis and An Illustrative Example
  9. Optimal Computation Load Problem
  10. Storage Placement Problem
  11. Computation Assignment Problem
  12. General Solutions for USCTEC with Storage Constraints
  13. Discussions

Key Result

Theorem 1

When $l=L+S$ and $\boldsymbol{\sigma} = \boldsymbol{e} = \boldsymbol{1}$, the optimal computation load vector, induced by the solution to problem eq-opt-single, is the solution to $(L+S, \boldsymbol{s}, \boldsymbol{1})$-LP, without considering an explicit storage placement and computation assignment

Figures (1)

  • Figure 1: Storage Placement and Storage Selections in Example \ref{['ex-general']}: The $x$-axis represents machine labels. The $y$-axis represents the location of rows in $\boldsymbol{A}$. The union of red and purple bars represents the storage selection for $\boldsymbol{s}_1$. The union of blue and purple bars represents the storage selection for $\boldsymbol{s}_2$. The purple bars represent the common storage selection for both $\boldsymbol{s}_1$ and $\boldsymbol{s}_2$. The red line at $y = \frac{3}{5}$ indicates a storage overflow occurred on machine $1$.

Theorems & Definitions (12)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Example 1
  • Definition 6
  • Theorem 1
  • Definition 7
  • Theorem 2
  • ...and 2 more