Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hybrid CG-Tikhonov is a filtration of the CG Lanczos vectors

Daniel Gerth, Kirk M. Soodhalter

TL;DR

The paper addresses regularization for ill-posed linear problems by linking CGne and CGtikh through a Lanczos-based filtration of Krylov directions. It develops explicit representations showing CGtikh acts as a damped version of CGne within the same Lanczos basis, with damping factors governed by polynomials in the Tikhonov parameter c. The authors derive shift-invariance relations via inverses of shifted tridiagonal matrices and extend the analysis to Golub-Kahan bidiagonalization in infinite dimensions, culminating in a general filtration framework. They illustrate the approach with numerical examples, discuss residual interpretations and stopping criteria, and outline future directions for parameter selection and broader hybrid methods.

Abstract

We consider iterative methods for solving linear ill-posed problems with compact operator and right-hand side only available via noise-polluted measurements. Conjugate gradients (CG) applied to the normal equations with an appropriate stopping rule and CG applied to the system solving for a Tikhonov-regularized solution (CGT) $(A^\ast A + c I_{\mathcal{X}}) x^{(δ,c)} = A^\ast y^δ$ are closely related regularization methods that build iterates from the same Krylov subspaces. In this work, we show that the CGT iterate can be expressed as $ x^{(δ,c)}_m = \sum_{i=1}^{m} γ^{(m)}_i(c) z_i^{(m)}v_i, $ where $\left\lbraceγ_i^{(m)}(c)\right\rbrace_{i=1}^m$ are functions of the Tikhonov parameter $c$ and $x^{(δ)}_m = \sum_{i=1}^{m} z_i^{(m)}v_i$ is the $m$-th CG iterate. We call these functions Lanczos filters, and they can be shown to have decay properties as $c\rightarrow\infty$ with the speed of decay increasing with $i$. This has the effect of filtering out the contribution of the later terms of the CG iterate. The filters can be constructed using quantities defined via recursions at each iteration. We demonstrate with numerical experiments that good parameter choices correspond to appropriate damping of the Lanczos vectors. The filtration approach also provides a platform for further development of parameter choice rules, and similar representations may hold for other hybrid iterative schemes.

Hybrid CG-Tikhonov is a filtration of the CG Lanczos vectors

TL;DR

The paper addresses regularization for ill-posed linear problems by linking CGne and CGtikh through a Lanczos-based filtration of Krylov directions. It develops explicit representations showing CGtikh acts as a damped version of CGne within the same Lanczos basis, with damping factors governed by polynomials in the Tikhonov parameter c. The authors derive shift-invariance relations via inverses of shifted tridiagonal matrices and extend the analysis to Golub-Kahan bidiagonalization in infinite dimensions, culminating in a general filtration framework. They illustrate the approach with numerical examples, discuss residual interpretations and stopping criteria, and outline future directions for parameter selection and broader hybrid methods.

Abstract

We consider iterative methods for solving linear ill-posed problems with compact operator and right-hand side only available via noise-polluted measurements. Conjugate gradients (CG) applied to the normal equations with an appropriate stopping rule and CG applied to the system solving for a Tikhonov-regularized solution (CGT) are closely related regularization methods that build iterates from the same Krylov subspaces. In this work, we show that the CGT iterate can be expressed as where are functions of the Tikhonov parameter and is the -th CG iterate. We call these functions Lanczos filters, and they can be shown to have decay properties as with the speed of decay increasing with . This has the effect of filtering out the contribution of the later terms of the CG iterate. The filters can be constructed using quantities defined via recursions at each iteration. We demonstrate with numerical experiments that good parameter choices correspond to appropriate damping of the Lanczos vectors. The filtration approach also provides a platform for further development of parameter choice rules, and similar representations may hold for other hybrid iterative schemes.

Paper Structure

This paper contains 19 sections, 12 theorems, 73 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Organization and notation of this paper
  3. Regularization and filter functions
  4. Self-adjoint Lanczos, CG and, CGtikh
  5. Self-adjoint Lanczos
  6. Conjugate Gradients via Lanczos
  7. Hybrid CGtikh via Lanczos
  8. Filter functions of CG and CGtikh
  9. Filter functions of CGtikh
  10. On the inverse of a shifted tri-diagonal
  11. Golub-Kahan Bidiagonalization and Lanczos in infinite dimensions
  12. GKb in the infinite dimensional setting
  13. Filtration of the Lanczos vectors
  14. Expressing CGtikh as a filtration of the CGne Lanczos directions
  15. A tale of two residuals
  16. ...and 4 more sections

Key Result

proposition 1

Denoting by $({\boldsymbol{T}}_m ^{-1})_{ij}\in\mathbb{R}$ the $(ij)$th entry of ${\boldsymbol{T}}_m ^{-1}$ with diagonal entries $a_i$ and super-/sub-diagonal entries $b_i$ defined as in the Lanczos recurrence eqn:Lanczos-basis-matrix, we can write where $\theta_\ell$ and $\phi_\ell^{(m)}$ satisfy the recurrence relations with the initial values $\theta_0 =1$ and $\theta_1 =a_1$, and with the

Figures (7)

  • Figure 1: Solution, right-hand side, and noise vectors for $\mathtt{shaw\_chebfun()}$
  • Figure 2: For the $\mathtt{shaw(400)}$ test problem with relative noise-level $10^{-4}$, a comparison of the true solution to the regularized solutions produced by Tikhonov with theoretical optimal parameter $c_{opt}$ and CG stopped using the noise-based discrepancy principle.
  • Figure 3: The absolute values of the Lanczos vector coefficients of CGne(30), CGne stopped using the descrepancy principle, and CGtikh(30) using $c=c_{opt}$ from O’Leary:2001:1. It should be noted that the coefficients from discrepancy principle CG are indeed close to the first seven coefficients of the CGtikh(30) iterate.
  • Figure 4: Lanczos filters for discrepancy principle iterate $m_{discr}=7$ and CGtikh$c=c_{opt}$ with respect to CGne(30) iterate.
  • Figure 5: Demonstration of the Lanczos filter dampening effect.
  • ...and 2 more figures

Theorems & Definitions (26)

  • remark 1
  • proposition 1: adapted from DaFonseca:2007:1
  • corollary 1
  • proof
  • lemma 1
  • proof
  • corollary 2
  • corollary 3
  • corollary 4
  • proof
  • ...and 16 more