Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A heavy-ball type curve search method for smooth convexly constrained optimization

Federica Donnini, Pierluigi Mansueto

Abstract

This paper addresses smooth convexly constrained optimization problems where the Euclidean projection onto the feasible set is computationally tractable. Although momentum techniques like Polyak's heavy-ball are known for accelerating optimization algorithms, their use in constrained settings remains limited due to challenges in preserving feasibility and ensuring convergence. We thus propose a heavy-ball-type method that extends to the constrained case a recently introduced curve-search globalization framework. The method attempts a momentum update and performs a curvilinear search to enforce an Armijo-type descent condition: when the momentum step is infeasible or unacceptable, the algorithm smoothly reverts to a feasible descent direction. We prove that the algorithm is well-defined and globally convergent to stationary points; the derivation of these results is nontrivial due to the use of a heavy-ball type direction in a constrained setting, where it may generate infeasible iterates. We discuss the incorporation of further mechanisms into the algorithm, including non-monotone curve search, spectral steplength selection and an adaptive momentum strategy. Numerical experiments on benchmark problems show the method is robust and competitive with the state-of-the-art.

A heavy-ball type curve search method for smooth convexly constrained optimization

Abstract

This paper addresses smooth convexly constrained optimization problems where the Euclidean projection onto the feasible set is computationally tractable. Although momentum techniques like Polyak's heavy-ball are known for accelerating optimization algorithms, their use in constrained settings remains limited due to challenges in preserving feasibility and ensuring convergence. We thus propose a heavy-ball-type method that extends to the constrained case a recently introduced curve-search globalization framework. The method attempts a momentum update and performs a curvilinear search to enforce an Armijo-type descent condition: when the momentum step is infeasible or unacceptable, the algorithm smoothly reverts to a feasible descent direction. We prove that the algorithm is well-defined and globally convergent to stationary points; the derivation of these results is nontrivial due to the use of a heavy-ball type direction in a constrained setting, where it may generate infeasible iterates. We discuss the incorporation of further mechanisms into the algorithm, including non-monotone curve search, spectral steplength selection and an adaptive momentum strategy. Numerical experiments on benchmark problems show the method is robust and competitive with the state-of-the-art.
Paper Structure (16 sections, 9 theorems, 37 equations, 6 figures, 1 table, 3 algorithms)

This paper contains 16 sections, 9 theorems, 37 equations, 6 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. The Curve Search Method for Unconstrained Optimization
  4. A Curvilinear Descent Method for Constrained Optimization
  5. Algorithmic Scheme
  6. Convergence Analysis
  7. Further Developments of the Curve Search Method for Constrained Optimization
  8. Non-monotone Curve Search
  9. Spectral Curve Search Method
  10. Adaptive Momentum Strategy
  11. Computational Experiments
  12. Experimental Settings
  13. Performance Overview
  14. Conclusions
  15. Supplementary Mathematical Proofs
  16. ...and 1 more sections

Key Result

Lemma 2.5

Let $\gamma:[0,1]\to\mathbb{R}^n$ be the quadratic Bézier curve corresponding to points $(P_0,P_1,P_2)$ and $\hat{t}\in[0,1]$. Then, for all $t\in[0,\hat{t}]$, $\gamma(t)\in \text{CH}(P_0,P_1,\gamma(\hat{t}))$.

Figures (6)

  • Figure 1: Feasible curve $\gamma_k$ of the form \ref{['eq::quadratic_gamma']} satisfying Assumption \ref{['ass::dk_feas']}. The convex feasible set $\Omega$ is highlighted in yellow.
  • Figure 2: Two case studies of quadratic curves of the form \ref{['eq::quadratic_gamma']} on the two-dimensional problem with one constraint described in Example \ref{['ex::example']}.
  • Figure 3: Graphical representation of the curves considered by Algorithm \ref{['alg::CSM_constrained']} in the two scenarios described in Example \ref{['ex::example']}.
  • Figure 4: Performance profiles obtained by SCS and SPG for both the non-monotone version with $M=10$ and the monotone one with respect to $T$ on the problems of Table \ref{['tab::problems']}. Note that the intervals of the x-axes were set to improve the visualization of the numerical results. For further details on the computation of the performance profiles, we refer the reader to Section \ref{['sec::experiments']}.
  • Figure 5: Performance profiles obtained by SCS and SPG for both the non-monotone version with $M=10$ and the monotone one with respect to $T$ on the problems of Table \ref{['tab::problems']} whose solution lies on the boundary of the feasible set. Note that the intervals of the x-axes were set to improve the visualization of the numerical results. For further details on the computation of the performance profiles, we refer the reader to Section \ref{['sec::experiments']}.
  • ...and 1 more figures

Theorems & Definitions (26)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3: bertsekas1999nonlinear
  • Definition 2.4: donnini2025efficientglobalizationheavyballtype
  • Lemma 2.5
  • Lemma 2.6
  • Definition 3.1
  • Definition 3.2
  • Proposition 3.3
  • proof
  • ...and 16 more