Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Projected Newton method for large-scale Bayesian linear inverse problems

Haibo Li

TL;DR

The paper addresses the regularized solution of Bayesian linear inverse problems and simultaneous estimation of the regularization parameter. It reformulates the Bayes problem as a noise-constrained minimization with the Lagrangian $\mathcal{L}(x,\lambda)$ and shows the correspondence $\mu = 1/\lambda$, enabling joint estimation of $x$ and $\lambda$. It introduces the Projected Newton method (PNT) that uses generalized Golub–Kahan bidiagonalization to form Krylov subspaces and compute a projected Newton direction, avoiding high-cost inversions and relying on small-scale solves. The work establishes existence and uniqueness of the constrained solution and its multiplier, demonstrates rigorous convergence, and shows strong robustness and efficiency on both small and large-scale Bayesian inverse problems, where the main cost is matrix–vector products.

Abstract

Computing the regularized solution of Bayesian linear inverse problems as well as the corresponding regularization parameter is highly desirable in many applications. This paper proposes a novel iterative method, termed the Projected Newton method (PNT), that can simultaneously update the regularization parameter and solution step by step without requiring any high-cost matrix inversions or decompositions. By reformulating the Tikhonov regularization as a constrained minimization problem and writing its Lagrangian function, a Newton-type method coupled with a Krylov subspace method, called the generalized Golub-Kahan bidiagonalization, is employed for the unconstrained Lagrangian function. The resulting PNT algorithm only needs solving a small-scale linear system to get a descent direction of a merit function at each iteration, thus significantly reducing computational overhead. Rigorous convergence results are proved, showing that PNT always converges to the unique regularized solution and the corresponding Lagrangian multiplier. Experimental results on both small and large-scale Bayesian inverse problems demonstrate its excellent convergence property, robustness and efficiency. Given that the most demanding computational tasks in PNT are primarily matrix-vector products, it is particularly well-suited for large-scale problems.

Projected Newton method for large-scale Bayesian linear inverse problems

TL;DR

The paper addresses the regularized solution of Bayesian linear inverse problems and simultaneous estimation of the regularization parameter. It reformulates the Bayes problem as a noise-constrained minimization with the Lagrangian and shows the correspondence , enabling joint estimation of and . It introduces the Projected Newton method (PNT) that uses generalized Golub–Kahan bidiagonalization to form Krylov subspaces and compute a projected Newton direction, avoiding high-cost inversions and relying on small-scale solves. The work establishes existence and uniqueness of the constrained solution and its multiplier, demonstrates rigorous convergence, and shows strong robustness and efficiency on both small and large-scale Bayesian inverse problems, where the main cost is matrix–vector products.

Abstract

Computing the regularized solution of Bayesian linear inverse problems as well as the corresponding regularization parameter is highly desirable in many applications. This paper proposes a novel iterative method, termed the Projected Newton method (PNT), that can simultaneously update the regularization parameter and solution step by step without requiring any high-cost matrix inversions or decompositions. By reformulating the Tikhonov regularization as a constrained minimization problem and writing its Lagrangian function, a Newton-type method coupled with a Krylov subspace method, called the generalized Golub-Kahan bidiagonalization, is employed for the unconstrained Lagrangian function. The resulting PNT algorithm only needs solving a small-scale linear system to get a descent direction of a merit function at each iteration, thus significantly reducing computational overhead. Rigorous convergence results are proved, showing that PNT always converges to the unique regularized solution and the corresponding Lagrangian multiplier. Experimental results on both small and large-scale Bayesian inverse problems demonstrate its excellent convergence property, robustness and efficiency. Given that the most demanding computational tasks in PNT are primarily matrix-vector products, it is particularly well-suited for large-scale problems.
Paper Structure (3 sections, 2 theorems, 15 equations)

This paper contains 3 sections, 2 theorems, 15 equations.

Table of Contents

  1. Introduction
  2. Noise constrained minimization for Bayesian inverse problems
  3. Noise constrained minimization

Key Result

Theorem 2.1

\newlabelthm:lagr0 The noise constrained minimization discrepancy has a unique solution $\boldsymbol{x}^{*}$ satisfying $\|\boldsymbol{A}\boldsymbol{x}^{*}-\boldsymbol{b}\|_{\boldsymbol{M}^{-1}}^{2}=\tau m$. Furthermore, there is a unique $\lambda^{*}>0$, which is the Lagrangian multiplier corresp

Theorems & Definitions (4)

  • Theorem 2.1
  • Proof 1
  • Lemma 2.2
  • Proof 2