Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nonmonotone subgradient methods based on a local descent lemma

Francisco J. Aragón-Artacho, Rubén Campoy, Pedro Pérez-Aros, David Torregrosa-Belén

TL;DR

This work develops a nonmonotone subgradient framework for nonsmooth, nonconvex optimization with upper-$\mathcal{C}^2$ objectives, culminating in the Self-adaptive Nonmonotone Subgradient Method (SNSM). It proves subsequential convergence to stationary points and demonstrates the method's flexibility to incorporate Newton/quasi-Newton directions, along with a concrete implementation for the minimum sum-of-squares clustering problem. Through numerical experiments against DCA, iDCA, BDCA, and RCSN on clustering and quadratic-integer tasks, SNSM shows competitive or superior convergence speed and solution quality, leveraging nonmonotone linesearch and auto-tuning. Overall, the paper broadens the algorithmic toolkit for nonsmooth optimization by enabling effective linesearch-based schemes for upper-$\mathcal{C}^2$ functions with practical data-mining applications.

Abstract

The aim of this paper is to extend the context of nonmonotone descent methods to the class of nonsmooth and nonconvex functions called upper-$\mathcal{C}^2$, which satisfy a nonsmooth and local version of the descent lemma. Under this assumption, we propose a general subgradient method that performs a nonmonotone linesearch, and we prove subsequential convergence to a stationary point of the optimization problem. Our approach allows us to cover the setting of various subgradient algorithms, including Newton and quasi-Newton methods. In addition, we propose a specification of the general scheme, named Self-adaptive Nonmonotone Subgradient Method (SNSM), which automatically updates the parameters of the linesearch. Particular attention is paid to the minimum sum-of-squares clustering problem, for which we provide a concrete implementation of SNSM. We conclude with some numerical experiments where we exhibit the advantages of SNSM in comparison with some known algorithms.

Nonmonotone subgradient methods based on a local descent lemma

TL;DR

This work develops a nonmonotone subgradient framework for nonsmooth, nonconvex optimization with upper- objectives, culminating in the Self-adaptive Nonmonotone Subgradient Method (SNSM). It proves subsequential convergence to stationary points and demonstrates the method's flexibility to incorporate Newton/quasi-Newton directions, along with a concrete implementation for the minimum sum-of-squares clustering problem. Through numerical experiments against DCA, iDCA, BDCA, and RCSN on clustering and quadratic-integer tasks, SNSM shows competitive or superior convergence speed and solution quality, leveraging nonmonotone linesearch and auto-tuning. Overall, the paper broadens the algorithmic toolkit for nonsmooth optimization by enabling effective linesearch-based schemes for upper- functions with practical data-mining applications.

Abstract

The aim of this paper is to extend the context of nonmonotone descent methods to the class of nonsmooth and nonconvex functions called upper-, which satisfy a nonsmooth and local version of the descent lemma. Under this assumption, we propose a general subgradient method that performs a nonmonotone linesearch, and we prove subsequential convergence to a stationary point of the optimization problem. Our approach allows us to cover the setting of various subgradient algorithms, including Newton and quasi-Newton methods. In addition, we propose a specification of the general scheme, named Self-adaptive Nonmonotone Subgradient Method (SNSM), which automatically updates the parameters of the linesearch. Particular attention is paid to the minimum sum-of-squares clustering problem, for which we provide a concrete implementation of SNSM. We conclude with some numerical experiments where we exhibit the advantages of SNSM in comparison with some known algorithms.
Paper Structure (14 sections, 4 theorems, 74 equations, 3 figures, 3 tables, 2 algorithms)

This paper contains 14 sections, 4 theorems, 74 equations, 3 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Keywords
  2. MSC2020
  3. Introduction
  4. Preliminaries and notation
  5. Upper-$\mathcal{C}^2$ functions and the descent lemma
  6. A nonmonotone subgradient method for nonconvex minimization
  7. Self-adaptive setting of stepsize and memory parameters
  8. Minimum sum-of-squares clustering problem
  9. Numerical experiments
  10. Parameter tuning
  11. Quadratic optimization with integer constraints
  12. Minimum Sum-of-Squares Clustering Problem
  13. Conclusion
  14. Acknowledgements

Key Result

Proposition 3.2

Let $U$ be an open set such that $\varphi:\mathbb{R}^n\to\mathbb{R}$ is locally Lipschitz on $U$. The following assertions are equivalent:

Figures (3)

  • Figure 1: Illustration of the function $\varphi_{MSC}$ for $m=3$, $s=1$ and $\ell=2$, with $(a^1,a^2,a^3)=(-1,0,1)$
  • Figure 2: Iterations generated by the algorithms (top) and function values (bottom) for two different problems and starting points (left and right), both with balls of radii $\frac{2}{20}\sqrt{2}$
  • Figure 3: Objective values attained by each of the algorithms for $n=50$ using $100$ random starting points, for different values of the radii of the balls. The mean value of each algorithm is represented by a black point and the lowest mean by a dotted line

Theorems & Definitions (16)

  • Definition 3.1: Upper-$\mathcal{C}^2$ functions
  • Proposition 3.2
  • proof
  • Example 3.3: Operations over upper-$\mathcal{C}^2$ functions
  • Example 3.4: Difference of a $\mathcal{C}^{1,+}$ function and a convex function
  • Example 3.5: Moreau and forward-backward envelopes
  • Example 3.6: Augmented Lagrangian method
  • Remark 4.2: On the choice of directions $d_k$
  • Proposition 4.3
  • proof
  • ...and 6 more