Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semi-Monotone Goldstein Line Search Strategy with Application in Sparse Recovery

Shima Shabani, Michael Breuß

TL;DR

This paper tackles unconstrained convex optimization with a smooth term $f$ and a non-smooth regularizer $c$, exemplified by basis pursuit denoising with $F(x)=\frac{1}{2}\|Ax-b\|^2+\mu\|x\|_1$. It introduces a semi-monotone Goldstein line search that relaxes the upper descent bound to allow larger step sizes while preserving a lower bound, and proves global convergence to a stationary point with local $R$-linear convergence under strong convexity. The authors provide a detailed convergence theory and validate the method, named smISGA, through extensive compressed sensing experiments where it competes favorably with state-of-the-art solvers. The work offers a robust, efficient alternative for large-scale sparse recovery problems and suggests avenues for extending the framework to broader convex-nonsmooth optimization settings.

Abstract

Line search methods are a prominent class of iterative methods to solve unconstrained minimization problems. These methods produce new iterates utilizing a suitable step size after determining proper directions for minimization. In this paper we propose a semi-monotone line search technique based on the Goldstein quotient for dealing with convex non-smooth optimization problems. The method allows to employ large step sizes away from the optimum thus improving the efficacy compared to standard Goldstein approach. For the presented line search method, we prove global convergence to a stationary point and local R-linear convergence rate in strongly convex cases. We report on some experiments in compressed sensing. By comparison with several state-of-the-art algorithms in the field, we demonstrate the competitive performance of the proposed approach and specifically its high efficiency.

Semi-Monotone Goldstein Line Search Strategy with Application in Sparse Recovery

TL;DR

This paper tackles unconstrained convex optimization with a smooth term and a non-smooth regularizer , exemplified by basis pursuit denoising with . It introduces a semi-monotone Goldstein line search that relaxes the upper descent bound to allow larger step sizes while preserving a lower bound, and proves global convergence to a stationary point with local -linear convergence under strong convexity. The authors provide a detailed convergence theory and validate the method, named smISGA, through extensive compressed sensing experiments where it competes favorably with state-of-the-art solvers. The work offers a robust, efficient alternative for large-scale sparse recovery problems and suggests avenues for extending the framework to broader convex-nonsmooth optimization settings.

Abstract

Line search methods are a prominent class of iterative methods to solve unconstrained minimization problems. These methods produce new iterates utilizing a suitable step size after determining proper directions for minimization. In this paper we propose a semi-monotone line search technique based on the Goldstein quotient for dealing with convex non-smooth optimization problems. The method allows to employ large step sizes away from the optimum thus improving the efficacy compared to standard Goldstein approach. For the presented line search method, we prove global convergence to a stationary point and local R-linear convergence rate in strongly convex cases. We report on some experiments in compressed sensing. By comparison with several state-of-the-art algorithms in the field, we demonstrate the competitive performance of the proposed approach and specifically its high efficiency.

Paper Structure

This paper contains 15 sections, 5 theorems, 42 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Review of Important Techniques
  3. Semi-Monotone Goldstein Strategy
  4. Convergence Analysis
  5. Bounded Gain in the Target Function
  6. Reaching to a Stationary Point
  7. R-Linear Local Convergence
  8. Numerical Experiments
  9. Experimental Setting
  10. Implementation Details
  11. Evaluation of Computational Efficiency
  12. Evaluation of Quality
  13. Conclusion on Experiments
  14. Conclusion
  15. Acknowledgments.

Key Result

theorem thmcountertheorem

If for the descent direction $d_k$, the value $\alpha_k$ satisfies the sufficient descent condition e:sufdescom and in addition is satisfied, then for any step size $\alpha_k'$ with $F(x_k+\alpha_k' d_k)\leq F_{k+1}$ we have

Figures (2)

  • Figure 1: Comparison among FISTA, FPC-BB, SpaRSA, TwIST, and smISGA with the performance cost metric CPU(Sec), nIter, and nFun, respectively. $x$ and $y$ axes indicate $\varsigma$ and $P_{{\tt s}}(\varsigma)$, respectively; see \ref{['e:perfo']}.
  • Figure 2: Scatter plot to show relationships between the (logarithmic) relative errors of ISGA and smISGA over all test problems. We observe that in a high percentage of test problems the relative error obtained by smISGA is in comparison much lower than for ISGA. This confirms the usefulness of the proposed semi-monotone strategy to help in convergence.

Theorems & Definitions (12)

  • theorem thmcountertheorem
  • proof
  • remark thmcounterremark
  • theorem thmcountertheorem
  • lemma thmcounterlemma
  • proof
  • remark thmcounterremark
  • lemma thmcounterlemma
  • proof
  • remark thmcounterremark
  • ...and 2 more