Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hybrid LSMR algorithms for large-scale general-form regularization

Yanfei Yang

TL;DR

The paper tackles large-scale ill-posed linear systems by developing a hybrid LSMR method that projects the problem onto Krylov subspaces via Golub–Kahan bidiagonalization and then applies a general-form regularization to the projected problems. It solves the inner least-squares problems with LSQR, with stopping rules to guarantee that the regularized solution obtained from iterative inner solves matches that from exact inner solves, and the inner problem conditioning improves as the subspace grows, accelerating convergence. The projected operator satisfies $B_{proj}^{\dagger}B_{proj}=Q_kQ_k^T$, and the outer solution is $x_{L,k}=x_k - (L(I_n-Q_kQ_k^T))^{\dagger}Lx_k$. Numerical experiments on 1D and 2D problems from the regularization toolbox show that hyb-LSMR matches or exceeds the accuracy of the JBDQR method while providing substantial speedups, even on large-scale instances up to $m=n=65{,}536$.

Abstract

The hybrid LSMR algorithm is proposed for large-scale general-form regularization. It is based on a Krylov subspace projection method where the matrix $A$ is first projected onto a subspace, typically a Krylov subspace, which is implemented via the Golub-Kahan bidiagonalization process applied to $A$, with starting vector $b$. Then a regularization term is employed to the projections. Finally, an iterative algorithm is exploited to solve a least squares problem with constraints. The resulting algorithms are called the {hybrid LSMR algorithm}. At every step, we exploit LSQR algorithm to solve the inner least squares problem, which is proven to become better conditioned as the number of $k$ increases, so that the LSQR algorithm converges faster. We prove how to select the stopping tolerances for LSQR in order to guarantee that the regularized solution obtained by iteratively computing the inner least squares problems and the one obtained by exactly computing the inner least squares problems have the same accuracy. Numerical experiments illustrate that the best regularized solution by the hybrid LSMR algorithm is as accurate as that by JBDQR which is a joint bidiagonalization based algorithm.

Hybrid LSMR algorithms for large-scale general-form regularization

TL;DR

The paper tackles large-scale ill-posed linear systems by developing a hybrid LSMR method that projects the problem onto Krylov subspaces via Golub–Kahan bidiagonalization and then applies a general-form regularization to the projected problems. It solves the inner least-squares problems with LSQR, with stopping rules to guarantee that the regularized solution obtained from iterative inner solves matches that from exact inner solves, and the inner problem conditioning improves as the subspace grows, accelerating convergence. The projected operator satisfies , and the outer solution is . Numerical experiments on 1D and 2D problems from the regularization toolbox show that hyb-LSMR matches or exceeds the accuracy of the JBDQR method while providing substantial speedups, even on large-scale instances up to .

Abstract

The hybrid LSMR algorithm is proposed for large-scale general-form regularization. It is based on a Krylov subspace projection method where the matrix is first projected onto a subspace, typically a Krylov subspace, which is implemented via the Golub-Kahan bidiagonalization process applied to , with starting vector . Then a regularization term is employed to the projections. Finally, an iterative algorithm is exploited to solve a least squares problem with constraints. The resulting algorithms are called the {hybrid LSMR algorithm}. At every step, we exploit LSQR algorithm to solve the inner least squares problem, which is proven to become better conditioned as the number of increases, so that the LSQR algorithm converges faster. We prove how to select the stopping tolerances for LSQR in order to guarantee that the regularized solution obtained by iteratively computing the inner least squares problems and the one obtained by exactly computing the inner least squares problems have the same accuracy. Numerical experiments illustrate that the best regularized solution by the hybrid LSMR algorithm is as accurate as that by JBDQR which is a joint bidiagonalization based algorithm.
Paper Structure (4 sections, 2 theorems, 37 equations, 2 figures, 3 tables, 3 algorithms)

This paper contains 4 sections, 2 theorems, 37 equations, 2 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Golub-Kahan bidiagonalization process and LSMR
  3. Hybrid LSMR algorithm
  4. Numerical experiments

Key Result

Theorem 3.1

\newlabelthm0 Let $Q_{k}$ and $Q_{k+1}$ be defined in lsqr1. Then the solution to hyb can be written as

Figures (2)

  • Figure 1: The relative error of hyb-LSMR and JBDQR with $\varepsilon=10^{-2}$: (a) baart; (b) shaw; (c) heat; (d) gravity; (e) Gaussianblur440; (f) grain; (g) text2; (h) satellite.
  • Figure 2: The exact images and the reconstructed images for the four two dimensional test problems with $\varepsilon=10^{-2}$ and $L$ defined in (\ref{['l3']}): (a) Gaussianblur440; (b) grain;(c) text2; (d) satellite.

Theorems & Definitions (4)

  • Theorem 3.1
  • Proof 1
  • Theorem 3.2
  • Proof 2