Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Straggler-tolerant stationary methods for linear systems

Vassilis Kalantzis, Yuanzhe Xi, Lior Horesh, Yousef Saad

TL;DR

This paper proposes straggler-tolerant Richardson iteration scheme and Chebyshev semi-iterative schemes and proves sufficient conditions for their convergence in expectation, and demonstrates the effectiveness of the proposed schemes on a few sparse matrix problems.

Abstract

In this paper, we consider the iterative solution of linear algebraic equations under the condition that matrix-vector products with the coefficient matrix are computed only partially. At the same time, non-computed entries are set to zeros. We assume that both the number of computed entries and their associated row index set are random variables, with the row index set sampled uniformly given the number of computed entries. This model of computations is realized in hybrid cloud computing architectures following the controller-worker distributed model under the influence of straggling workers. We propose straggler-tolerant Richardson iteration scheme and Chebyshev semi-iterative schemes, and prove sufficient conditions for their convergence in expectation. Numerical experiments verify the presented theoretical results as well as the effectiveness of the proposed schemes on a few sparse matrix problems.

Straggler-tolerant stationary methods for linear systems

TL;DR

This paper proposes straggler-tolerant Richardson iteration scheme and Chebyshev semi-iterative schemes and proves sufficient conditions for their convergence in expectation, and demonstrates the effectiveness of the proposed schemes on a few sparse matrix problems.

Abstract

In this paper, we consider the iterative solution of linear algebraic equations under the condition that matrix-vector products with the coefficient matrix are computed only partially. At the same time, non-computed entries are set to zeros. We assume that both the number of computed entries and their associated row index set are random variables, with the row index set sampled uniformly given the number of computed entries. This model of computations is realized in hybrid cloud computing architectures following the controller-worker distributed model under the influence of straggling workers. We propose straggler-tolerant Richardson iteration scheme and Chebyshev semi-iterative schemes, and prove sufficient conditions for their convergence in expectation. Numerical experiments verify the presented theoretical results as well as the effectiveness of the proposed schemes on a few sparse matrix problems.
Paper Structure (16 sections, 6 theorems, 49 equations, 9 figures, 1 algorithm)

This paper contains 16 sections, 6 theorems, 49 equations, 9 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. A model for partially complete matrix-vector products
  3. The problem of straggling workers
  4. Matrix-vector products with omitted entries
  5. Relation to asynchronous models
  6. Richardson iteration with straggling workers
  7. A straggler-tolerant scheme
  8. Convergence Analysis
  9. Effects of the iteration number $m$
  10. Chebyshev semi-iterative method with straggling workers
  11. Numerical Experiments
  12. Illustration of Algorithm \ref{['alg:richardson']}
  13. Performance on general sparse matrices
  14. Performance on model problems
  15. Straggler-tolerant second-order iterations
  16. ...and 1 more sections

Key Result

Lemma 3.2

\newlabellem10 Let ${\@fontswitch{}{\mathcal{}} T}$ denote a random subset of $\{1,2,\ldots,N\}$ whose cardinality depends on the random integer variable $T$ that takes values from $1$ to $N$, where ${\@fontswitch{}{\mathcal{}} T}$ is sampled uniformly. Then,

Figures (9)

  • Figure 1: Matrix-vector multiplication $y=D_{\@fontswitch{}{\mathcal{}} T}Af$ under the controller-worker model for a toy example with $N=4$ and $T=2,\ {\@fontswitch{}{\mathcal{}} T}=\{1,4\}$.
  • Figure 1: Average sample variance of the entries of $\widehat{z}_m$ for a $N=10^3$ model Laplacian problem.
  • Figure 1: MSE of $\frac{1}{L}\sum_{i=1}^L\widehat{z}_m^{(i)}-z_m$ on up to $L=100$ trials of Algorithm \ref{['alg:richardson']} for the matrix crystm01 as well as MSE of the quantity $\frac{1}{L}\sum_{i=1}^L\widehat{z}_m^{(i)}-z$ where $z$ is the solution of $Az=v$.
  • Figure 2: Matrix-vector multiplication $y=D_{\@fontswitch{}{\mathcal{}} T}Af$ under the controller-worker model for a toy example with $N=4$ and $T=1,\ {\@fontswitch{}{\mathcal{}} T}=\{3\}$.
  • Figure 2: MSE of $\frac{1}{L}\sum_{i=1}^L\widehat{z}_m^{(i)}-z_m$ on up to $L=100$ trials of Algorithm \ref{['alg:richardson']} for the matrix bundle1 as well as MSE of the quantity $\frac{1}{L}\sum_{i=1}^L\widehat{z}_m^{(i)}-z$ where $z$ is the solution of $Az=v$.
  • ...and 4 more figures

Theorems & Definitions (15)

  • Definition 2.1
  • Remark 2.2
  • Remark 3.1
  • Lemma 3.2
  • Proof 1
  • Proposition 3.3
  • Proof 2
  • Lemma 3.4
  • Proof 3
  • Theorem 3.5
  • ...and 5 more