The Detection and Correction of Silent Errors in Pipelined Krylov Subspace Methods

TL;DR

This work tackles silent errors in large SPD linear system solves using pipelined Krylov methods by deriving finite-precision bounds on gaps between predicted and recomputed quantities in Pipe-PR-CG. The authors introduce ν-gap, w-gap, and μ-gap criteria, plus a relative μ-gap–bound difference, to detect faults, and they couple these with a rollback-based recovery (FT-Pipe-PR-CG). They further improve practicality with an adaptive threshold (AFT-Pipe-PR-CG) to reduce false positives while preserving fault detection. The approach enables robust, low-overhead fault tolerance for parallel Krylov solvers, with demonstrated effectiveness in detecting faults that would otherwise delay convergence, and clear guidance for applying these methods alongside preconditioning in future work.

Abstract

As computational machines become larger and more complex, the probability of hardware failure rises. ``Silent errors'', or bit flips, may not be immediately apparent but can cause detrimental effects to algorithm behavior. In this work, we examine an algorithm-based approach to silent error detection in the context of pipelined Krylov subspace methods, in particular, Pipe-PR-CG, for the solution of linear systems. Our approach is based on using finite precision error analysis to bound the differences between quantities which should be equal in exact arithmetic. By monitoring select quantities during the iteration, we can detect when these bounds are violated, which indicates that a silent error has occurred. We use this approach to develop a fault-tolerant variant and also suggest a strategy for dynamically adapting the detection criteria. Our numerical experiments demonstrate the effectiveness of our approach.

Figures (13)

• Figure 1: Bit flip sensitivity: averages from all variables.
• Figure 2: Convergence of the relative residual $\mathbf{||r_k||/||b||}$ when the 15th bit of $\beta_k$ is flipped for matrix 1138_bus. Purple dotted lines denote flip iterations.
• Figure 3: Residual convergence curves when various bits of $\sigma_k$ are flipped in the 100th iteration for the matrix bundle1.
• Figure 4: The $\nu$-gap ($|\nu_k - \nu_k'|$) when the 15th bit of $\gamma_k$ is flipped in the 100th iteration for the matrix bcsstm07
• Figure 5: $\nu$-gap (red) and $\nu$-gap bound (blue) graph, matrix bcsstm07