Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Anderson acceleration with approximate calculations: applications to scientific computing

Massimiliano Lupo Pasini, M. Paul Laiu

TL;DR

This work develops rigorous bounds for Anderson acceleration (AA) that permit approximate calculations in linear fixed-point problems, showing convergence is preserved when perturbations stay within a computable budget. It introduces a reduced variant, Reduced Alternating AA, which projects the least-squares step onto a dynamically sized subspace of dimension $s$, with $s$ chosen by practical heuristics and monitored by a residual-monotonicity check. The approach also includes two row-selection strategies (Subselected and Randomized) to drive the subspace projection, and analyzes backward stability for both LHS and RHS perturbations. Numerical experiments on linear Richardson-type iterations and nonlinear Boltzmann fixed-point problems demonstrate that approximate calculations can substantially reduce computational cost while maintaining accuracy, highlighting potential HPC benefits. The results suggest that adaptive accuracy control and projection-based LS reduction are viable pathways to scalable, robust fixed-point solvers for both linear and nonlinear scientific computing problems.

Abstract

We provide rigorous theoretical bounds for Anderson acceleration (AA) that allow for approximate calculations when applied to solve linear problems. We show that, when the approximate calculations satisfy the provided error bounds, the convergence of AA is maintained while the computational time could be reduced. We also provide computable heuristic quantities, guided by the theoretical error bounds, which can be used to automate the tuning of accuracy while performing approximate calculations. For linear problems, the use of heuristics to monitor the error introduced by approximate calculations, combined with the check on monotonicity of the residual, ensures the convergence of the numerical scheme within a prescribed residual tolerance. Motivated by the theoretical studies, we propose a reduced variant of AA, which consists in projecting the least-squares used to compute the Anderson mixing onto a subspace of reduced dimension. The dimensionality of this subspace adapts dynamically at each iteration as prescribed by the computable heuristic quantities. We numerically show and assess the performance of AA with approximate calculations on: (i) linear deterministic fixed-point iterations arising from the Richardson's scheme to solve linear systems with open-source benchmark matrices with various preconditioners and (ii) non-linear deterministic fixed-point iterations arising from non-linear time-dependent Boltzmann equations.

Anderson acceleration with approximate calculations: applications to scientific computing

TL;DR

This work develops rigorous bounds for Anderson acceleration (AA) that permit approximate calculations in linear fixed-point problems, showing convergence is preserved when perturbations stay within a computable budget. It introduces a reduced variant, Reduced Alternating AA, which projects the least-squares step onto a dynamically sized subspace of dimension , with chosen by practical heuristics and monitored by a residual-monotonicity check. The approach also includes two row-selection strategies (Subselected and Randomized) to drive the subspace projection, and analyzes backward stability for both LHS and RHS perturbations. Numerical experiments on linear Richardson-type iterations and nonlinear Boltzmann fixed-point problems demonstrate that approximate calculations can substantially reduce computational cost while maintaining accuracy, highlighting potential HPC benefits. The results suggest that adaptive accuracy control and projection-based LS reduction are viable pathways to scalable, robust fixed-point solvers for both linear and nonlinear scientific computing problems.

Abstract

We provide rigorous theoretical bounds for Anderson acceleration (AA) that allow for approximate calculations when applied to solve linear problems. We show that, when the approximate calculations satisfy the provided error bounds, the convergence of AA is maintained while the computational time could be reduced. We also provide computable heuristic quantities, guided by the theoretical error bounds, which can be used to automate the tuning of accuracy while performing approximate calculations. For linear problems, the use of heuristics to monitor the error introduced by approximate calculations, combined with the check on monotonicity of the residual, ensures the convergence of the numerical scheme within a prescribed residual tolerance. Motivated by the theoretical studies, we propose a reduced variant of AA, which consists in projecting the least-squares used to compute the Anderson mixing onto a subspace of reduced dimension. The dimensionality of this subspace adapts dynamically at each iteration as prescribed by the computable heuristic quantities. We numerically show and assess the performance of AA with approximate calculations on: (i) linear deterministic fixed-point iterations arising from the Richardson's scheme to solve linear systems with open-source benchmark matrices with various preconditioners and (ii) non-linear deterministic fixed-point iterations arising from non-linear time-dependent Boltzmann equations.
Paper Structure (17 sections, 5 theorems, 61 equations, 4 figures, 4 tables, 1 algorithm)

This paper contains 17 sections, 5 theorems, 61 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Fixed-point iteration
  3. Anderson acceleration and the Alternating Anderson acceleration
  4. Stationary Richardson
  5. Anderson acceleration for linear systems
  6. Computational cost of AAR
  7. Backward stability analysis of approximate least-squares solves for AAR
  8. Backward stability analysis of AAR with perturbed LHS
  9. Backward stability analysis of AAR with perturbed LHS and RHS
  10. Reduced Alternating AA
  11. Computational cost of Reduced Alternating AA for linear fixed-point iterations
  12. Numerical results
  13. Linear deterministic fixed-point iteration: iterative linear solver
  14. Injection of random noise on LHS of least-squares for each AA step
  15. Reduced Alternating AA to solve sparse linear systems
  16. ...and 2 more sections

Key Result

lemma 1

Assume that $k={z}p$ iterations of Full AAR have been carried out. Let ${\hat{\mathbf{g}}}^{k}$ be the Anderson mixing computed by solving the perturbed least-squares problem defined in Equation least_squares_perturbed_matrix. Then, the following inequality holds

Figures (4)

  • Figure 1: Approximate AA convergence history.
  • Figure 2: Performance comparison between iterative solvers on sparse linear systems. Here the performance metric $P$ given in \ref{['eq:performance_metric']} is reported for each solver $s$ with varying $\tau$. All AA variants use the full history ($m=k$).
  • Figure 3: Performance comparison between iterative solvers on sparse linear systems. Here the performance metric $P$ given in \ref{['eq:performance_metric']} is reported for each solver $s$ with varying $\tau$. All AA variants use a history truncated to $m=20$.
  • Figure 4: Iteration counts and computation time (in $\log$ scale) for various fixed-point solvers for solving the non-linear system \ref{['eq:fixed_point_Boltzmann']} at different matter density, which corresponds to the stiffness of the system.

Theorems & Definitions (15)

  • lemma 1
  • proof
  • theorem 1
  • proof
  • remark 1
  • remark 2
  • corollary 1
  • remark 3
  • remark 4
  • remark 5
  • ...and 5 more