Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conjugate gradient for ill-posed problems: regularization by preconditioning, preconditioning by regularization

Ahmed Chabib, Jean-Francois Witz, Vincent Magnier, Pierre Gosselet

TL;DR

This work advances the use of preconditioned conjugate gradient methods for ill-posed problems by intimately linking preconditioning with Tikhonov regularization through a shared operator M. By leveraging Ritz analysis, it enables costless exploration of multiple regularization weights, informed by L-curve and Picard diagnostics, and supports a posteriori filtering and subspace recycling to accelerate sequences of regularized solves. The methodology is demonstrated on linear data completion via Steklov-Poincaré formulations and on nonlinear optical-flow recovery with matrix-free implementation, achieving robust reconstructions and efficient reuse of spectral information. The results highlight the practical impact of regularization-aware preconditioning for both linear and nonlinear ill-posed problems, with clear paths to extending the approach to inexact preconditioners and broader problem classes.

Abstract

This paper investigates using the conjugate gradient iterative solver for ill-posed problems. We show that preconditioner and Tikhonov-regularization work in conjunction. In particular when they employ the same symmetric positive semi-definite operator, a powerful Ritz analysis allows one to estimate at negligible computational cost the solution for any Tikhonov's weight. This enhanced linear solver is applied to the boundary data completion problem and as the inner solver for the optical flow estimator.

Conjugate gradient for ill-posed problems: regularization by preconditioning, preconditioning by regularization

TL;DR

This work advances the use of preconditioned conjugate gradient methods for ill-posed problems by intimately linking preconditioning with Tikhonov regularization through a shared operator M. By leveraging Ritz analysis, it enables costless exploration of multiple regularization weights, informed by L-curve and Picard diagnostics, and supports a posteriori filtering and subspace recycling to accelerate sequences of regularized solves. The methodology is demonstrated on linear data completion via Steklov-Poincaré formulations and on nonlinear optical-flow recovery with matrix-free implementation, achieving robust reconstructions and efficient reuse of spectral information. The results highlight the practical impact of regularization-aware preconditioning for both linear and nonlinear ill-posed problems, with clear paths to extending the approach to inexact preconditioners and broader problem classes.

Abstract

This paper investigates using the conjugate gradient iterative solver for ill-posed problems. We show that preconditioner and Tikhonov-regularization work in conjunction. In particular when they employ the same symmetric positive semi-definite operator, a powerful Ritz analysis allows one to estimate at negligible computational cost the solution for any Tikhonov's weight. This enhanced linear solver is applied to the boundary data completion problem and as the inner solver for the optical flow estimator.
Paper Structure (24 sections, 44 equations, 10 figures, 3 tables, 2 algorithms)

This paper contains 24 sections, 44 equations, 10 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preconditioned Conjugate Gradient and Ritz elements
  3. Appropriate solution and error norms and stopping criteria
  4. Natural frame for the L-curve during iterations
  5. A posteriori filtrering
  6. Preconditioning by regularization
  7. Assessment in the linear case: data completion problem
  8. Laplace PDE with missing and redundant boundaries
  9. Steklov-Poincaré approach
  10. Direct solvers
  11. Preconditioned iterative solvers
  12. Opportunities when preconditioning with Tikhonov operator
  13. Augmented preconditioned conjugate gradient for sequences of regularized systems
  14. Roles of augmentation
  15. Postprocessing of multiple solutions for various regularization
  16. ...and 9 more sections

Figures (10)

  • Figure 1: Solution to be identified by Steklov-Poincaré approach observing only the left-hand side (indiscernible oscillation), for $k=3$ and $H=T=1$.
  • Figure 2: Data completion problem: quality of solution using CG with various preconditioner -- effect of slight Tikhonov regularization on the worst preconditioner.
  • Figure 3: L-curve obtained with direct solves vs L-curve estimated by Ritz' approximation \ref{['eq:l-curves2']} after one iterative resolution with $\lambda=10^{-9}$ materialized by the dot $(\epsilon=10^{-9})$. The vertical asymptotes are aligned with the $y$-axis.
  • Figure 4: Picard plot processed after the iterative resolution with $\lambda=10^{-9}$ and $\varepsilon=10^{-12}$.
  • Figure 5: Speckle of the test specimen.
  • ...and 5 more figures

Theorems & Definitions (1)

  • Remark 1