An iterative algorithm for the square-root Lasso

TL;DR

Problem: sparse reconstruction with a scale-invariant square-root Lasso objective $\Phi_{\mu}(f)=\|Af-g\|+\mu\|f\|_1$ in linear inverse problems. Approach: derive SQRT-ISTA by introducing the auxiliary residual $\sigma=\|Af-g\|$ and a majorisation-minimisation surrogate, yielding the adaptive-threshold update $f_{k+1}=S_{2\tau\mu\sigma_k}(f_k+\tau A^*(g-Af_k))$ with $\sigma_{k+1}=\|Af_{k+1}-g\|$, and prove convergence under mild step-size conditions; extend to the group square-root Lasso. Contributions: rigorous convergence analysis (including Kurdyka–Łojasiewicz arguments) and explicit rate bounds under standard assumptions; a simple, implementable first-order method that aligns with ISTA but with residual-driven thresholds. Significance: provides a straightforward solver for square-root Lasso in inverse problems that can handle ill-conditioned or compact operators, with a scalable extension to grouped sparsity and clearer parameter-tuning implications than traditional Lasso.

Abstract

In the framework of sparsity-enforcing regularisation for linear inverse problems, we consider the minimisation of a square-root Lasso cost function. To solve this problem we devise a simple modification (called SQRT-ISTA) of the Iterative Soft-Thresholding Algorithm (ISTA) for the Lasso problem and we prove convergence for this algorithm. Under some additional assumptions, we derive an upper bound on the convergence rate of the cost function. We also generalise these results to the case of the group square-root Lasso, where sparsity is enforced for groups of variables instead of individual ones.

Figures (1)

• Figure 1: Panels (a) and (b): Contour plots of the cost functions for the Lasso and square-root-Lasso problems for $A=(2\ 1)$ and $g=(2)$ and $\mu= 1$, $\tilde{\mu} = 2$. The respective minimizers are indicated with a thick black dot or line. The set $\{f \hbox{with} Af=g\}$ is shown with a dashed red line. Panels (c) and (d): A schematic representation of the convergence paths of the ISTA algorithm and of SQRT-ISTA algorithm (\ref{['iterates_tau_summ']},\ref{['minsig2_summ']}). While the ISTA algorithm always converges to a minimizer of the Lasso cost function, the SQRT-ISTA algorithm (\ref{['iterates_tau_summ']},\ref{['minsig2_summ']}) can also converge to a point on the red dashed line, for which $\sigma=0$.

Theorems & Definitions (20)

• Remark 1
• Proposition 1
• Lemma 1
• Lemma 2
• Proposition 2
• Proposition 3
• Proposition 4
• Proposition 5
• Proposition 6
• Remark 2