Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the determination of Lagrange Multipliers for a weighted LASSO problem using geometric and convex analysis techniques

Gianluca Giacchi, Bastien Milani, Benedetta Franchieschiello

TL;DR

This work addresses the problem of determining explicit Lagrange multipliers for a constrained, voxel-weighted LASSO in compressed sensing and MRI contexts. It leverages convex analysis, Lagrange duality, and subdifferential calculus to relate multipliers to inequality constraints and derives explicit expressions under structured assumptions, notably when $A^T A$ is diagonal or when gradient signs impose bounds. A key result in the scalar case yields $\lambda^#=2\|a_{\ast}\|_2^2\Big(\frac{|\langle b, a_{\ast}\rangle|}{\|a_{\ast}\|_2^2}-\tau\Big)^+$, and the analysis extends to decoupled coordinates and explicit solutions via $(A^T A+\Delta_\lambda)x^#=A^Tb$. The findings enable direct use of Lagrange multipliers as voxel-wise tuning parameters for denoising and MR image reconstruction, while outlining open problems for non-diagonal $A^T A$ and generalized inner products that are relevant for non-Cartesian sampling in MRI.

Abstract

Compressed Sensing (CS) encompasses a broad array of theoretical and applied techniques for recovering signals, given partial knowledge of their coefficients. Its applications span various fields, including mathematics, physics, engineering, and several medical sciences. Motivated by our interest in the mathematics behind Magnetic Resonance Imaging (MRI) and CS, we employ convex analysis techniques to analytically determine equivalents of Lagrange multipliers for optimization problems with inequality constraints, specifically a weighted LASSO with voxel-wise weighting. We investigate this problem under assumptions on the fidelity term $\Vert{Ax-b}\Vert_2^2$, either concerning the sign of its gradient or orthogonality-like conditions of its matrix. To be more precise, we either require the sign of each coordinate of $2(Ax-b)^TA$ to be fixed within a rectangular neighborhood of the origin, with the side lengths of the rectangle dependent on the constraints, or we assume $A^TA$ to be diagonal. The objective of this work is to explore the relationship between Lagrange multipliers and the constraints of a weighted variant of LASSO, specifically in the mentioned cases where this relationship can be computed explicitly. As they scale the regularization terms of the weighted LASSO, Lagrange multipliers serve as tuning parameters for the weighted LASSO, prompting the question of their potential effective use as tuning parameters in applications like MR image reconstruction and denoising. This work represents an initial step in this direction.

On the determination of Lagrange Multipliers for a weighted LASSO problem using geometric and convex analysis techniques

TL;DR

This work addresses the problem of determining explicit Lagrange multipliers for a constrained, voxel-weighted LASSO in compressed sensing and MRI contexts. It leverages convex analysis, Lagrange duality, and subdifferential calculus to relate multipliers to inequality constraints and derives explicit expressions under structured assumptions, notably when is diagonal or when gradient signs impose bounds. A key result in the scalar case yields , and the analysis extends to decoupled coordinates and explicit solutions via . The findings enable direct use of Lagrange multipliers as voxel-wise tuning parameters for denoising and MR image reconstruction, while outlining open problems for non-diagonal and generalized inner products that are relevant for non-Cartesian sampling in MRI.

Abstract

Compressed Sensing (CS) encompasses a broad array of theoretical and applied techniques for recovering signals, given partial knowledge of their coefficients. Its applications span various fields, including mathematics, physics, engineering, and several medical sciences. Motivated by our interest in the mathematics behind Magnetic Resonance Imaging (MRI) and CS, we employ convex analysis techniques to analytically determine equivalents of Lagrange multipliers for optimization problems with inequality constraints, specifically a weighted LASSO with voxel-wise weighting. We investigate this problem under assumptions on the fidelity term , either concerning the sign of its gradient or orthogonality-like conditions of its matrix. To be more precise, we either require the sign of each coordinate of to be fixed within a rectangular neighborhood of the origin, with the side lengths of the rectangle dependent on the constraints, or we assume to be diagonal. The objective of this work is to explore the relationship between Lagrange multipliers and the constraints of a weighted variant of LASSO, specifically in the mentioned cases where this relationship can be computed explicitly. As they scale the regularization terms of the weighted LASSO, Lagrange multipliers serve as tuning parameters for the weighted LASSO, prompting the question of their potential effective use as tuning parameters in applications like MR image reconstruction and denoising. This work represents an initial step in this direction.
Paper Structure (18 sections, 18 theorems, 154 equations)

This paper contains 18 sections, 18 theorems, 154 equations.

Table of Contents

  1. Introduction
  2. Preliminaries and notation
  3. Notation.
  4. Lagrange duality
  5. Subdifferential
  6. Lagrange formulation of constrained problems
  7. The weighted LASSO
  8. The scalar case
  9. Properties of $\mathcal{A}$
  10. A result under conditions on the gradient of $\left\lVert Ax-b\right\rVert_2^2$
  11. Decoupling the variables
  12. Explicit solution
  13. Considerations and conclusions
  14. Applications
  15. Open problems
  16. ...and 3 more sections

Key Result

Proposition 2.3

Let $\Omega\subseteq\mathds{R}^n$ be open and $g:\Omega\to\mathds{R}$ be convex and continuous on $\Omega$. Let $x_0\in\Omega$. Then, $\partial g(x_0)\neq\varnothing$.

Theorems & Definitions (47)

  • Definition 2.1
  • Definition 2.2: Subdifferential
  • Proposition 2.3
  • Theorem 2.4: Cf. BV, Section 5.3.2
  • Definition 2.5: Separating hyperplane
  • Theorem 2.6: Cf. R3 Theorem 3.4
  • proof : Idea of the proof of Theorem \ref{['thm1']}
  • Definition 2.7: Lagrange Multipliers
  • Corollary 2.8: Cf. FR Theorem B.28
  • proof
  • ...and 37 more