Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Parameter Selection by GCV and a $χ^2$ test within Iterative Methods for $\ell_1$-regularized Inverse Problems

Brian Sweeney, Rosemary Renaut, Malena Español

TL;DR

The paper tackles ill-posed inverse problems with $\ell_1$ regularization by analyzing Split Bregman and Majorization-Minimization methods that convert the problem into a series of inner $\ell_2$-regularized subproblems. It develops and extends parameter selection strategies—generalized cross validation and $\chi^2$ dof tests—for these inner problems, including a new $p>n$ chi-squared result and a means to incorporate prior information via the $A$-weighted generalized inverse. Numerical experiments on 1D and 2D image deblurring show that adapting the inner regularization parameter per iteration yields final reconstructions comparable to using an optimal fixed $\lambda$, often achieving improvement in early iterations and allowing early fixation of $\lambda$ to reduce cost. The framework demonstrates robust performance across SB and MM, with GCV and central $\chi^2$ typically offering strong accuracy and efficiency, while DP and RWP are less competitive, especially in higher-dimensional problems.

Abstract

$\ell_1$ regularization is used to preserve edges or enforce sparsity in a solution to an inverse problem. We investigate the Split Bregman and the Majorization-Minimization iterative methods that turn this non-smooth minimization problem into a sequence of steps that include solving an $\ell_2$-regularized minimization problem. We consider selecting the regularization parameter in the inner generalized Tikhonov regularization problems that occur at each iteration in these $\ell_1$ iterative methods. The generalized cross validation and $χ^2$ degrees of freedom methods are extended to these inner problems. In particular, for the $χ^2$ method this includes extending the $χ^2$ result for problems in which the regularization operator has more rows than columns, and showing how to use the $A-$weighted generalized inverse to estimate prior information at each inner iteration. Numerical experiments for image deblurring problems demonstrate that it is more effective to select the regularization parameter automatically within the iterative schemes than to keep it fixed for all iterations. Moreover, an appropriate regularization parameter can be estimated in the early iterations and used fixed to convergence.

Parameter Selection by GCV and a $χ^2$ test within Iterative Methods for $\ell_1$-regularized Inverse Problems

TL;DR

The paper tackles ill-posed inverse problems with regularization by analyzing Split Bregman and Majorization-Minimization methods that convert the problem into a series of inner -regularized subproblems. It develops and extends parameter selection strategies—generalized cross validation and dof tests—for these inner problems, including a new chi-squared result and a means to incorporate prior information via the -weighted generalized inverse. Numerical experiments on 1D and 2D image deblurring show that adapting the inner regularization parameter per iteration yields final reconstructions comparable to using an optimal fixed , often achieving improvement in early iterations and allowing early fixation of to reduce cost. The framework demonstrates robust performance across SB and MM, with GCV and central typically offering strong accuracy and efficiency, while DP and RWP are less competitive, especially in higher-dimensional problems.

Abstract

regularization is used to preserve edges or enforce sparsity in a solution to an inverse problem. We investigate the Split Bregman and the Majorization-Minimization iterative methods that turn this non-smooth minimization problem into a sequence of steps that include solving an -regularized minimization problem. We consider selecting the regularization parameter in the inner generalized Tikhonov regularization problems that occur at each iteration in these iterative methods. The generalized cross validation and degrees of freedom methods are extended to these inner problems. In particular, for the method this includes extending the result for problems in which the regularization operator has more rows than columns, and showing how to use the weighted generalized inverse to estimate prior information at each inner iteration. Numerical experiments for image deblurring problems demonstrate that it is more effective to select the regularization parameter automatically within the iterative schemes than to keep it fixed for all iterations. Moreover, an appropriate regularization parameter can be estimated in the early iterations and used fixed to convergence.
Paper Structure (21 sections, 2 theorems, 64 equations, 11 figures, 2 tables, 2 algorithms)

This paper contains 21 sections, 2 theorems, 64 equations, 11 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Iterative methods for l1 regularization
  3. The Split Bregman method
  4. The Majorization-Minimization method
  5. The inner minimization problem
  6. Matrix decompositions
  7. Estimation of the regularization parameter
  8. The method of generalized cross validation
  9. The chi-squared degrees of freedom test
  10. The non-central chi-squared dof test
  11. Other Methods
  12. Numerical examples
  13. A 1D example
  14. Split Bregman
  15. Majorization-Minimization
  16. ...and 6 more sections

Key Result

Theorem 1

\newlabelThm:gcv0 The GCV function for eq:genTik_hk, where ${\bf h}^{(k)}$ is replaced by ${\bf h}$, has the same form as GCVGen, but now with ${\bf x}_\lambda$ given by where ${\bf L}^\sharp_\lambda = \lambda^2{\bf A}_{\bf L}^{-1}{\bf L}^\top$. When $m \geq n$ and the GSVD is defined as in gsvd, $G(\lambda)$ for $n \geq p$ and $p>n$ is given by where we ignore any sum in which the lower limit

Figures (11)

  • Figure 1: Plots of $F(\lambda)$ and $F_C(\lambda)$ for MM, where the selected $\lambda$ is marked. In \ref{['fig:FvFC:b']}, $F_C(\lambda)$ is not monotonic and does not have a root.
  • Figure 1: \ref{['fig:xTrue']}: ${\bf x}$ and $\tilde{{\bf b}}$ for the 1D deblurring problem ($\text{SNR} = 20$). \ref{['fig:SBna:a', 'fig:MMna:a']}show the SB and MM solutions where $\lambda$ is fixed at the optimal.
  • Figure 2: Results for SB applied to \ref{['fig:xTrue']} where $\lambda$ is fixed at the optimal $\lambda=232.6$, or selected at each iteration with GCV or the $\chi^2$ dof tests. \ref{['fig:SBn:a']} plots the $\text{RE}$ by iteration, \ref{['fig:SBn:b']} plots the relative change in ${\bf x}$, \ref{['fig:SBn:c']} plots the $\lambda$ selected, \ref{['fig:SBn:d']} plots the relative change in $\lambda^2$, and \ref{['fig:SBn:e']} plots the ISNR.
  • Figure 3: SB solutions at convergence for \ref{['fig:xTrue']}.
  • Figure 4: Results for MM applied to \ref{['fig:xTrue']} where $\lambda$ is fixed at the optimal $\lambda=464.2$, or selected at each iteration with GCV or the $\chi^2$ dof tests. \ref{['fig:MMn:a']} plots the $\text{RE}$ by iteration, \ref{['fig:MMn:b']} plots the relative change in ${\bf x}$, \ref{['fig:MMn:c']} plots the $\lambda$ selected, \ref{['fig:MMn:d']} plots the relative change in $\lambda^2$, and \ref{['fig:MMn:e']} plots the ISNR.
  • ...and 6 more figures

Theorems & Definitions (4)

  • Theorem 1
  • Proof 1
  • Theorem 2
  • Proof 2