Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A survey of scalar and vector extrapolation

Khalide Jbilou

TL;DR

This survey analyzes convergence acceleration via scalar and vector extrapolation methods, tracing historical roots to Aitken, Richardson, Shanks, and Wynn and detailing their connections to Padé approximants. It then extends these ideas to vector, block, and topological variants (MPE, RRE, MMPE, VEA, TEA, Anderson), showing equivalences to Krylov subspace methods and applicability to linear systems, nonlinear fixed‑point iterations, and ill‑posed problems. The paper reinforces the practical value with numerical experiments on a nonlinear PDE, Google PageRank, and Fredholm integrals, illustrating substantial reductions in iterations and compute time. By providing determinant/Schur‑complement representations and discussing stability, restart strategies, and block generalizations, the work offers a cohesive framework to select and implement effective extrapolation tools in large‑scale numerical computations.

Abstract

Scalar extrapolation and convergence acceleration methods are central tools in numerical analysis for improving the efficiency of iterative algorithms and the summation of slowly convergent series. These methods construct transformed sequences that converge more rapidly to the same limit without altering the underlying iterative process, thereby reducing computational cost and enhancing numerical accuracy. Historically, the origins of such techniques can be traced back to classical algebraic methods by AlKhwarizmi and early series acceleration techniques by Newton, while systematic approaches emerged in the 20th century with Aitken process and Richardson extrapolation. Later developments, including the Shanks transformation and Wynn epsilon algorithm, provided general frameworks capable of eliminating multiple dominant error components, with deep connections to Pade approximants and rational approximations of generating functions. This paper presents a comprehensive review of classical scalar extrapolation methods, including Richardson extrapolation, Aitken process, Shanks transformation, Wynn epsilon algorithm, and other algorithms. We examine their theoretical foundations, asymptotic error models, convergence properties, numerical stability, and practical implementation considerations. The second part of this work is dedicated to vector extrapolation methods: polynomial based ones and epsilon algorithm generalizations to vector sequences. Additionally, we highlight modern developments such as their applications to iterative solvers, Krylov subspace methods, and large-scale computational simulations. The aim of this review is to provide a unified perspective on scalar and vector extrapolation techniques, bridging historical origins, theoretical insights, and contemporary computational applications.

A survey of scalar and vector extrapolation

TL;DR

This survey analyzes convergence acceleration via scalar and vector extrapolation methods, tracing historical roots to Aitken, Richardson, Shanks, and Wynn and detailing their connections to Padé approximants. It then extends these ideas to vector, block, and topological variants (MPE, RRE, MMPE, VEA, TEA, Anderson), showing equivalences to Krylov subspace methods and applicability to linear systems, nonlinear fixed‑point iterations, and ill‑posed problems. The paper reinforces the practical value with numerical experiments on a nonlinear PDE, Google PageRank, and Fredholm integrals, illustrating substantial reductions in iterations and compute time. By providing determinant/Schur‑complement representations and discussing stability, restart strategies, and block generalizations, the work offers a cohesive framework to select and implement effective extrapolation tools in large‑scale numerical computations.

Abstract

Scalar extrapolation and convergence acceleration methods are central tools in numerical analysis for improving the efficiency of iterative algorithms and the summation of slowly convergent series. These methods construct transformed sequences that converge more rapidly to the same limit without altering the underlying iterative process, thereby reducing computational cost and enhancing numerical accuracy. Historically, the origins of such techniques can be traced back to classical algebraic methods by AlKhwarizmi and early series acceleration techniques by Newton, while systematic approaches emerged in the 20th century with Aitken process and Richardson extrapolation. Later developments, including the Shanks transformation and Wynn epsilon algorithm, provided general frameworks capable of eliminating multiple dominant error components, with deep connections to Pade approximants and rational approximations of generating functions. This paper presents a comprehensive review of classical scalar extrapolation methods, including Richardson extrapolation, Aitken process, Shanks transformation, Wynn epsilon algorithm, and other algorithms. We examine their theoretical foundations, asymptotic error models, convergence properties, numerical stability, and practical implementation considerations. The second part of this work is dedicated to vector extrapolation methods: polynomial based ones and epsilon algorithm generalizations to vector sequences. Additionally, we highlight modern developments such as their applications to iterative solvers, Krylov subspace methods, and large-scale computational simulations. The aim of this review is to provide a unified perspective on scalar and vector extrapolation techniques, bridging historical origins, theoretical insights, and contemporary computational applications.
Paper Structure (34 sections, 5 theorems, 234 equations, 1 figure, 3 tables, 5 algorithms)

This paper contains 34 sections, 5 theorems, 234 equations, 1 figure, 3 tables, 5 algorithms.

Table of Contents

  1. Historical background of extrapolation methods
  2. Scalar extrapolation methods
  3. Richardson extrapolation
  4. Aitken’s $\Delta^2$ process
  5. The classical $\Delta^2$ algorithm
  6. The iterated $\Delta^2$ method
  7. Shanks transformation
  8. Wynn's $\varepsilon$-algorithm
  9. Connection with Padé approximation
  10. Continued fractions and rational approximation
  11. The $E$-algorithm
  12. The $\rho$ algorithm
  13. The $\theta$-algorithm
  14. Vector extrapolation methods
  15. The polynomial-based methods
  16. ...and 19 more sections

Key Result

Theorem 3.1

$\forall n$, $E_k^{(n)}= S$ if and only if

Figures (1)

  • Figure 4.1: The relative errors

Theorems & Definitions (9)

  • remark 1
  • Theorem 3.1
  • Theorem 4.1
  • Theorem 4.2
  • Theorem 4.3
  • Theorem 4.4
  • remark 2
  • remark 3
  • remark 4