Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Block CG algorithms revisited

Petr Tichý, Gérard Meurant, Dorota Šimonová

TL;DR

This work clarifies the precise relationship between block Lanczos and block conjugate gradient (BCG) methods under exact arithmetic and full block rank, showing how block Lanczos coefficients can be recovered within BCG and how block Jacobi matrices arise in this context. It introduces practical, rank-deficiency-robust BCG variants, especially the Dubrulle-R (DR-BCG) and Dubrulle-P (DP-BCG) schemes, and demonstrates through numerical experiments that DR-BCG offers superior convergence and accuracy for multiple right-hand sides, both with and without preconditioning. The paper provides a coherent framework to compute block Lanczos coefficients inside BCG, preserves block Jacobi structure, and outlines future work on quadrature-based error bounds and broader variant exploration, with significant implications for efficient large-scale block linear solves. Overall, the results offer a robust, scalable approach to solving block SPD systems in finite precision, with practical guidance on variant selection and preconditioning.

Abstract

Our goal in this paper is to clarify the relationship between the block Lanczos and the block conjugate gradient (BCG) algorithms. Under the full rank assumption for the block vectors, we show the one-to-one correspondence between the algorithms. This allows, for example, the computation of the block Lanczos coefficients in BCG. The availability of block Jacobi matrices in BCG opens the door for further development, e.g., for error estimation in BCG based on (modified) block Gauss quadrature rules. Driven by the need to get a practical variant of the BCG algorithm well suited for computations in finite precision arithmetic, we also discuss some important variants of BCG due to Dubrulle. These variants avoid the troubles with a possible rank deficiency within the block vectors. We show how to incorporate preconditioning and computation of Lanczos coefficients into these variants. We hope to help clarify which variant of the block conjugate gradient algorithm should be used for computations in finite precision arithmetic. Numerical results illustrate the performance of different variants of BCG on some examples.

Block CG algorithms revisited

TL;DR

This work clarifies the precise relationship between block Lanczos and block conjugate gradient (BCG) methods under exact arithmetic and full block rank, showing how block Lanczos coefficients can be recovered within BCG and how block Jacobi matrices arise in this context. It introduces practical, rank-deficiency-robust BCG variants, especially the Dubrulle-R (DR-BCG) and Dubrulle-P (DP-BCG) schemes, and demonstrates through numerical experiments that DR-BCG offers superior convergence and accuracy for multiple right-hand sides, both with and without preconditioning. The paper provides a coherent framework to compute block Lanczos coefficients inside BCG, preserves block Jacobi structure, and outlines future work on quadrature-based error bounds and broader variant exploration, with significant implications for efficient large-scale block linear solves. Overall, the results offer a robust, scalable approach to solving block SPD systems in finite precision, with practical guidance on variant selection and preconditioning.

Abstract

Our goal in this paper is to clarify the relationship between the block Lanczos and the block conjugate gradient (BCG) algorithms. Under the full rank assumption for the block vectors, we show the one-to-one correspondence between the algorithms. This allows, for example, the computation of the block Lanczos coefficients in BCG. The availability of block Jacobi matrices in BCG opens the door for further development, e.g., for error estimation in BCG based on (modified) block Gauss quadrature rules. Driven by the need to get a practical variant of the BCG algorithm well suited for computations in finite precision arithmetic, we also discuss some important variants of BCG due to Dubrulle. These variants avoid the troubles with a possible rank deficiency within the block vectors. We show how to incorporate preconditioning and computation of Lanczos coefficients into these variants. We hope to help clarify which variant of the block conjugate gradient algorithm should be used for computations in finite precision arithmetic. Numerical results illustrate the performance of different variants of BCG on some examples.

Paper Structure

This paper contains 17 sections, 6 theorems, 78 equations, 3 figures, 8 algorithms.

Table of Contents

  1. Introduction
  2. Lanczos and CG
  3. Block Lanczos and block CG
  4. Block Lanczos algorithm
  5. Block CG algorithm
  6. The connection
  7. Recurrences for the normalized residual blocks
  8. The connection between $T_{k}$ and $\widetilde{T}_{k}$
  9. Block Lanczos coefficients in BCG
  10. Practical block CG variants
  11. The Dubrulle-R variant
  12. The Dubrulle-P variant
  13. Practical issues
  14. Preconditioning
  15. Stopping the algorithms
  16. ...and 2 more sections

Key Result

Lemma 1

Consider the quantities determined by Algorithm alg:BCG. The BCG residual blocks $r_{j}$, $j=1,\dots,k$, satisfy with $R_{k}\in\mathbb{R}^{n\times km},$$R_{k}\equiv\left[r_{0},\dots,r_{k-2},r_{k-1}\right]$, and the $km\times km$ block tridiagonal coefficient matrix The matrix $\widehat{T}_{k}$ can be written in the factorized form

Figures (3)

  • Figure 1: The convergence characteristic $\omega_{k}$ in HS-BCG (red dotted), DP-BCG (blue dashed), and DR-BCG (black solid) for bcsstk03 and $b\in\mathbb{R}^{n\times m}$, $m=1$ (top left), $m=2$ (top right), $m=4$ (bottom left), $m=6$ (bottom right).
  • Figure 2: The convergence characteristic $\omega_{k}$ in HS-BCG (red dotted), DP-BCG (blue dashed), and DR-BCG (black solid) for s3dkt3m2 and $b\in\mathbb{R}^{n\times m}$, $m=1$ (top left), $m=4$ (top right), $m=16$ (bottom left), $m=64$ (bottom right).
  • Figure 3: The convergence characteristic $\omega_{k}$ in DP-BCG (blue) and BF-BCG (red) for s3dkt3m2 and random $b\in\mathbb{R}^{n\times m}$ with $m=16$ (top part) and $m=64$ (bottom part).

Theorems & Definitions (12)

  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • ...and 2 more