Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data-driven geometric parameter optimization for PD-GMRES

Lennart Duvenbeck, Cedric Riethmüller, Christian Rohde

TL;DR

This work tackles the challenge of selecting restart parameters for restarted GMRES by employing a data-driven, quadtree-based optimization of the PD-GMRES controller. By pairing parameter dimensions and using a heuristic runtime model, the authors efficiently identify parameter sets that yield superior convergence across diverse matrices, outperforming GMRES$(m)$ and default PD-GMRES on most training and test cases. They also address problematic matrices with specialized PD-GMRES configurations and extend the algorithm with a bounded restart parameter, $m_{ ext{max}}$, to further boost performance in slow-converging scenarios. The results demonstrate practical impact for Krylov-subspace solvers in large-scale linear systems, while also outlining avenues for preconditioning, classification, and alternative quadtree criteria to enhance robustness.

Abstract

Restarted GMRES is a robust and widely used iterative solver for linear systems. The control of the restart parameter is a key task to accelerate convergence and to prevent the well-known stagnation phenomenon. We focus on the Proportional-Derivative GMRES (PD-GMRES), which has been derived using control-theoretic ideas in [Cuevas Núñez, Schaerer, and Bhaya (2018)] as a versatile method for modifying the restart parameter. Several variants of a quadtree-based geometric optimization approach are proposed to find a best choice of PD-GMRES parameters. We show that the optimized PD-GMRES performs well across a large number of matrix types and we observe superior performance as compared to major other GMRES-based iterative solvers. Moreover, we propose an extension of the PD-GMRES algorithm to further improve performance by controlling the range of values for the restart parameter.

Data-driven geometric parameter optimization for PD-GMRES

TL;DR

This work tackles the challenge of selecting restart parameters for restarted GMRES by employing a data-driven, quadtree-based optimization of the PD-GMRES controller. By pairing parameter dimensions and using a heuristic runtime model, the authors efficiently identify parameter sets that yield superior convergence across diverse matrices, outperforming GMRES and default PD-GMRES on most training and test cases. They also address problematic matrices with specialized PD-GMRES configurations and extend the algorithm with a bounded restart parameter, , to further boost performance in slow-converging scenarios. The results demonstrate practical impact for Krylov-subspace solvers in large-scale linear systems, while also outlining avenues for preconditioning, classification, and alternative quadtree criteria to enhance robustness.

Abstract

Restarted GMRES is a robust and widely used iterative solver for linear systems. The control of the restart parameter is a key task to accelerate convergence and to prevent the well-known stagnation phenomenon. We focus on the Proportional-Derivative GMRES (PD-GMRES), which has been derived using control-theoretic ideas in [Cuevas Núñez, Schaerer, and Bhaya (2018)] as a versatile method for modifying the restart parameter. Several variants of a quadtree-based geometric optimization approach are proposed to find a best choice of PD-GMRES parameters. We show that the optimized PD-GMRES performs well across a large number of matrix types and we observe superior performance as compared to major other GMRES-based iterative solvers. Moreover, we propose an extension of the PD-GMRES algorithm to further improve performance by controlling the range of values for the restart parameter.

Paper Structure

This paper contains 16 sections, 2 theorems, 7 equations, 9 figures, 8 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. The PD-GMRES algorithm
  3. Geometric optimization approach
  4. The quadtree method
  5. Application to PD-GMRES
  6. A heuristic runtime estimate
  7. Parameter space considerations
  8. Matrix averaging and quadtree resolution
  9. Numerical experiments for optimized PD-GMRES
  10. Examples of PD-GMRES and quadtree behavior
  11. Applying the optimization approach to training matrices
  12. Evaluating performance of optimized PD-GMRES on test matrices
  13. Classification and optimization of problematic matrices
  14. Evaluating a different parameter quadrant
  15. An algorithmic extension for PD-GMRES
  16. ...and 1 more sections

Key Result

Proposition 2.1

For PD-GMRES with $\alpha_p < 0$ and $\alpha_d \geq 0$, there exists a number $N \in \{0,\ldots,n(n+1)/2\}$ such that $x_N = x = M^{-1}b$.

Figures (9)

  • Figure 3.1: Quadtree visualization.
  • Figure 4.1: Example quadtrees and PD-GMRES iteration behavior for the $(\alpha_p$, $\alpha_d)$-space (top) and the $(m_{\min}, m_{\text{step}})$-space (bottom).
  • Figure 4.2: Average quadtrees calculated by the optimization sequence for the training matrices using $m_{\text{init}} = 10$. Final parameters for the optimized PD-GMRES are $m_{\text{init}} = 10, m_{\min} = 3, m_{\text{step}} = 10, \alpha_p = -0.625, \alpha_d = 4.375$. The top row depicts the $(\alpha_p,\alpha_d)$-space, and the bottom row the $(m_{\min},m_{\text{step}})$-space.
  • Figure 4.3: Comparison of PD-GMRES performance for training matrices, optimized using $m_{\text{init}} = 10, 20, 30$: $m_{\text{init}} = 10, m_{\min} = 3, m_{\text{step}} = 10, \alpha_p = -0.625, \alpha_d = 4.375$, geometric mean = $1$$m_{\text{init}} = 20, m_{\min} = 1, m_{\text{step}} = 11, \alpha_p = -0.625, \alpha_d = 3.125$, geometric mean = $1.0522$$m_{\text{init}} = 30, m_{\min} = 5, m_{\text{step}} = 5, \alpha_p = -0.625, \alpha_d = 5.625$, geometric mean = $1.0142$
  • Figure 4.5: GMRES(100) performance (logarithmically) over matrix condition. For the named problems with blue marker GMRES(100) outperformed the optimized PD-GMRES.
  • ...and 4 more figures

Theorems & Definitions (6)

  • Proposition 2.1
  • proof
  • Remark 2.2
  • Proposition 2.3
  • proof
  • Remark 2.4