Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Variable Projected Augmented Lagrangian Methods for Generalized Lasso Problems

Stefano Aleotti, Davide Bianchi, Florian Bossmann, Riley Yizhou Chen, Matthias Chung

TL;DR

This work extends variable projection to generalized nonlinear Lasso problems by forming a smooth reduced objective $f_{\text{proj}}(x)$ through optimal $y_z(x)$ minimization. It introduces a preconditioned variant (pvpal) that leverages a Gauss–Newton–type approximation to accelerate convergence, while preserving the simple VPAL structure. The authors prove convergence under mild assumptions for both standard VPAL and pvpal, and validate the approach with extensive linear and nonlinear inverse problem experiments in imaging, including deblurring, inpainting, CT, phase retrieval, and LIP-CAR. Across tasks, VPAL and especially pvpal deliver faster convergence and higher-quality reconstructions, demonstrating strong potential for large-scale nonlinear inverse problems with nonsmooth regularization. The results suggest significant practical impact for imaging applications, with avenues for adaptive preconditioning and broader nonconvex extensions.

Abstract

We introduce variable projected augmented Lagrangian (VPAL) methods for solving generalized nonlinear Lasso problems with improved speed and accuracy. By eliminating the nonsmooth variable via soft-thresholding, VPAL transforms the problem into a smooth reduced formulation. For linear models, we develop a preconditioned variant that mimics Newton-type updates and yields significant acceleration. We prove convergence guarantees for both standard and preconditioned VPAL under mild assumptions and show that variable projection leads to sharper convergence and higher solution quality. The method seamlessly extends to nonlinear inverse problems, where it outperforms traditional approaches in applications such as phase retrieval and contrast enhanced MRI (LIP-CAR). Across tasks including deblurring, inpainting, and sparse-view tomography, VPAL consistently delivers state-of-the-art reconstructions, positioning variable projection as a powerful tool for modern large-scale inverse problems.

Variable Projected Augmented Lagrangian Methods for Generalized Lasso Problems

TL;DR

This work extends variable projection to generalized nonlinear Lasso problems by forming a smooth reduced objective through optimal minimization. It introduces a preconditioned variant (pvpal) that leverages a Gauss–Newton–type approximation to accelerate convergence, while preserving the simple VPAL structure. The authors prove convergence under mild assumptions for both standard VPAL and pvpal, and validate the approach with extensive linear and nonlinear inverse problem experiments in imaging, including deblurring, inpainting, CT, phase retrieval, and LIP-CAR. Across tasks, VPAL and especially pvpal deliver faster convergence and higher-quality reconstructions, demonstrating strong potential for large-scale nonlinear inverse problems with nonsmooth regularization. The results suggest significant practical impact for imaging applications, with avenues for adaptive preconditioning and broader nonconvex extensions.

Abstract

We introduce variable projected augmented Lagrangian (VPAL) methods for solving generalized nonlinear Lasso problems with improved speed and accuracy. By eliminating the nonsmooth variable via soft-thresholding, VPAL transforms the problem into a smooth reduced formulation. For linear models, we develop a preconditioned variant that mimics Newton-type updates and yields significant acceleration. We prove convergence guarantees for both standard and preconditioned VPAL under mild assumptions and show that variable projection leads to sharper convergence and higher solution quality. The method seamlessly extends to nonlinear inverse problems, where it outperforms traditional approaches in applications such as phase retrieval and contrast enhanced MRI (LIP-CAR). Across tasks including deblurring, inpainting, and sparse-view tomography, VPAL consistently delivers state-of-the-art reconstructions, positioning variable projection as a powerful tool for modern large-scale inverse problems.

Paper Structure

This paper contains 17 sections, 4 theorems, 47 equations, 18 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Structure
  4. Background of Variable projected augmented Lagrangian (vpal)
  5. vpal for nonlinear problems
  6. Preconditioned vpal
  7. Linear Numerical Experiments
  8. Experiment 1: Deblurring Peppers
  9. Experiment 2: Inpainting
  10. Experiment 3: Computed Tomography (CT)
  11. Nonlinear Numerical Experiments
  12. Experiment 4: Phase retrieval in Ptychographic Imaging
  13. Step size choice
  14. Experimental setup
  15. vpal and pvpal evaluation
  16. ...and 2 more sections

Key Result

Lemma 3.1

Assume that $A\in C^0(\mathbb{R}^{n}, \mathbb{R}^m)$. Furthermore, assume that $f_{\mathrm{joint}}(x,y)$ is strictly convex and coercive for $\mu=0$. Then for arbitrary but fixed $z$ and $\mu,\lambda>0$, $f_{\mathrm{joint}}(x,y)$ is strictly convex and has a unique minimizer $(\hat{x},\hat{y})$.

Figures (18)

  • Figure 1: Experiment 1: (a) Original image, (b) PSF used in the deblurring process, and (c) resulting blurred and noisy observation.
  • Figure 2: Experiment 1: Comparison between vpal and pvpal methods. The first row shows the behavior of the RRE with respect to the number of iterations and the corresponding computation time (in seconds) using the optimal step size strategy. In contrast, the second row focuses on the linearized strategy.
  • Figure 3: Comparison of reconstructions produced by the vpal and pvpal methods under equal computation time conditions.
  • Figure 4: Experiment 1: Analysis of the step size behavior for both the linear and optimal strategies in the vpal. Step size for pvpal is constant $\alpha = 1$.
  • Figure 5: Experiment 2: (a) Original image, (b) Observed image
  • ...and 13 more figures

Theorems & Definitions (9)

  • Remark 1
  • Lemma 3.1
  • proof
  • Lemma 3.2
  • proof
  • Lemma 3.3
  • proof
  • Theorem 3.4
  • proof