Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sharp estimates for the convergence rate of Orthomin(k) for a class of linear systems

Andrei Draganescu, Florin Spinu

TL;DR

This work establishes a sharp and universal upper bound on the convergence rate of Orthomin($k$) for systems of the form $(I+\rho U)x=b$ with a unitary $U$, showing $q_n\le\rho$ and proving sharpness via diagonal-unitary examples where the asymptotic rate equals $\rho$. It further demonstrates that for certain normal matrices with spectrum on an ellipse, Orthomin($k$) with $k\ge2$ shares a common, faster asymptotic rate than Orthomin(1), while for the 1-step method the rate can be slower or equal depending on spectral geometry. A detailed analysis of Orthomin(1) is developed, including monotonicity of the residual ratio, explicit recurrences for the case $A=I+\rho U$ with unit-modulus eigenvalues, and precise conditions under which $q_n$ converges to $\rho$ or fails to do so. The results tie into PDE discretizations, illustrating the practical relevance of spectrum-driven rate estimates for truncated Krylov methods and guiding expectations about when higher-order Orthomin($k$) can outperform Orthomin(1).

Abstract

In this work we show that the convergence rate of Orthomin($k$) applied to systems of the form $(I+ρU) x = b$, where $U$ is a unitary operator and $0<ρ<1$, is less than or equal to $ρ$. Moreover, we give examples of operators $U$ and $ρ>0$ for which the asymptotic convergence rate of Orthomin($k$) is exactly $ρ$, thus showing that the estimate is sharp. While the systems under scrutiny may not be of great interest in themselves, their existence shows that, in general, Orthomin($k$) does not converge faster than Orthomin(1). Furthermore, we give examples of systems for which Orthomin($k$) has the same asymptotic convergence rate as Orthomin(2) for $k\ge 2$, but smaller than that of Orthomin(1). The latter systems are related to the numerical solution of certain partial differential equations.

Sharp estimates for the convergence rate of Orthomin(k) for a class of linear systems

TL;DR

This work establishes a sharp and universal upper bound on the convergence rate of Orthomin() for systems of the form with a unitary , showing and proving sharpness via diagonal-unitary examples where the asymptotic rate equals . It further demonstrates that for certain normal matrices with spectrum on an ellipse, Orthomin() with shares a common, faster asymptotic rate than Orthomin(1), while for the 1-step method the rate can be slower or equal depending on spectral geometry. A detailed analysis of Orthomin(1) is developed, including monotonicity of the residual ratio, explicit recurrences for the case with unit-modulus eigenvalues, and precise conditions under which converges to or fails to do so. The results tie into PDE discretizations, illustrating the practical relevance of spectrum-driven rate estimates for truncated Krylov methods and guiding expectations about when higher-order Orthomin() can outperform Orthomin(1).

Abstract

In this work we show that the convergence rate of Orthomin() applied to systems of the form , where is a unitary operator and , is less than or equal to . Moreover, we give examples of operators and for which the asymptotic convergence rate of Orthomin() is exactly , thus showing that the estimate is sharp. While the systems under scrutiny may not be of great interest in themselves, their existence shows that, in general, Orthomin() does not converge faster than Orthomin(1). Furthermore, we give examples of systems for which Orthomin() has the same asymptotic convergence rate as Orthomin(2) for , but smaller than that of Orthomin(1). The latter systems are related to the numerical solution of certain partial differential equations.

Paper Structure

This paper contains 14 sections, 14 theorems, 89 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Brief background
  3. The main examples
  4. An upper bound
  5. Sharpness of the upper bound
  6. Connection with numerical partial differential equations
  7. Further examples: normal matrices with spectra on ellipses
  8. Convergence analysis for Orthomin(1)
  9. Monotonocity of $q_n$
  10. The case $\mu_k=1+\rho\zeta_k$, $|\zeta_k|=1$, and $r_0\in\mathbb{C}^d$ arbitrary
  11. Non-convergence to $\rho$
  12. The case d=2
  13. Convergence of $q_n$ to $\rho$
  14. Numerical evidence supporting Conjecture \ref{['conj:o1tokexamples']}

Key Result

Theorem 1

Assume that $0\notin {\mathcal{F}}(A)$ and $\delta=\mathrm{dist}(0, {\mathcal{F}}(A))$. If $r_n$ is the $n^{\mathrm{th}}$ residual in the Orthomin($k$) iteration, then where $|\!| A |\!|$ is the $2$-norm of the matrix $A$.

Figures (6)

  • Figure 1: The comparative residual norms for Orthomin(k) ($k=1,2,3,4,5,10$): for Orthomin$(1)$$q_n$ exceeds $0.7902$, but for $k=2, 3, 4, 5, 10$ we note a convergence of $q_n$ to a value near $0.6891227$.
  • Figure 2: This is an example where $-\rho$ does not belong to $\operatorname{Hull}(\zeta_1,\dots, \zeta_d)$: $\rho=0.9,\ d=15$.
  • Figure 3: The case when $-\rho$ does not belong to $\operatorname{Hull}(\zeta_1,\dots, \zeta_d)$: $\rho=0.9,\ d=15$.
  • Figure 4: Convergence results for $q_n$ to $\rho$ for Orthomin$(k)$, $k\in \{1, 2, 3, 7, 11, 13\}$, $\rho\in \{0.2, 0.5, 0.8\}$, and $d=16$.
  • Figure 5: Convergence results for $q_n$ to $\rho$ for Orthomin$(k)$, $k\in \{1, 2, 3, 7, 11, 13\}$, $\rho\in \{0.2, 0.5, 0.8\}$, and $d=32$.
  • ...and 1 more figures

Theorems & Definitions (27)

  • Theorem 1
  • Theorem 2
  • proof
  • Conjecture 3
  • Conjecture 4
  • Lemma 5
  • proof
  • Proposition 6
  • proof
  • Lemma 7
  • ...and 17 more