Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Smooth Quasar-Convex Optimization with Constraints

David Martínez-Rubio

TL;DR

This work addresses the problem of accelerating optimization for smooth $\gamma$-quasar-convex functions under general convex constraints. It introduces an inexact accelerated proximal-point framework that leverages the Moreau envelope and a line-search mechanism to handle constraint inexactness, achieving a nearly optimal rate of $\widetilde{O}\left(\frac{1}{\gamma}\sqrt{\frac{L D^2}{\epsilon}}\right)$ first-order queries. The authors also provide analyses of unconstrained analogs via projected gradient descent and Frank-Wolfe in the constrained quasar-convex setting, establishing explicit unaccelerated rates. This work extends quasar-convex optimization to general convex domains and improves on prior Riemannian approaches, with broad implications for linear dynamical systems, generalized linear models, and geodesic optimization within constant-curvature spaces.

Abstract

Quasar-convex functions form a broad nonconvex class with applications to linear dynamical systems, generalized linear models, and Riemannian optimization, among others. Current nearly optimal algorithms work only in affine spaces due to the loss of one degree of freedom when working with general convex constraints. Obtaining an accelerated algorithm that makes nearly optimal $\widetilde{O}(1/(γ\sqrtε))$ first-order queries to a $γ$-quasar convex smooth function \emph{with constraints} was independently asked as an open problem in Martínez-Rubio (2022); Lezane, Langer, and Koolen (2024). In this work, we solve this question by designing an inexact accelerated proximal point algorithm that we implement using a first-order method achieving the aforementioned rate and, as a consequence, we improve the complexity of the accelerated geodesically Riemannian optimization solution in Martínez-Rubio (2022). We also analyze projected gradient descent and Frank-Wolfe algorithms in this constrained quasar-convex setting. To the best of our knowledge, our work provides the first analyses of first-order methods for quasar-convex smooth functions with general convex constraints.

Smooth Quasar-Convex Optimization with Constraints

TL;DR

This work addresses the problem of accelerating optimization for smooth -quasar-convex functions under general convex constraints. It introduces an inexact accelerated proximal-point framework that leverages the Moreau envelope and a line-search mechanism to handle constraint inexactness, achieving a nearly optimal rate of first-order queries. The authors also provide analyses of unconstrained analogs via projected gradient descent and Frank-Wolfe in the constrained quasar-convex setting, establishing explicit unaccelerated rates. This work extends quasar-convex optimization to general convex domains and improves on prior Riemannian approaches, with broad implications for linear dynamical systems, generalized linear models, and geodesic optimization within constant-curvature spaces.

Abstract

Quasar-convex functions form a broad nonconvex class with applications to linear dynamical systems, generalized linear models, and Riemannian optimization, among others. Current nearly optimal algorithms work only in affine spaces due to the loss of one degree of freedom when working with general convex constraints. Obtaining an accelerated algorithm that makes nearly optimal first-order queries to a -quasar convex smooth function \emph{with constraints} was independently asked as an open problem in Martínez-Rubio (2022); Lezane, Langer, and Koolen (2024). In this work, we solve this question by designing an inexact accelerated proximal point algorithm that we implement using a first-order method achieving the aforementioned rate and, as a consequence, we improve the complexity of the accelerated geodesically Riemannian optimization solution in Martínez-Rubio (2022). We also analyze projected gradient descent and Frank-Wolfe algorithms in this constrained quasar-convex setting. To the best of our knowledge, our work provides the first analyses of first-order methods for quasar-convex smooth functions with general convex constraints.

Paper Structure

This paper contains 19 sections, 10 theorems, 67 equations, 1 figure, 3 algorithms.

Table of Contents

  1. INTRODUCTION
  2. Main Contributions.
  3. PRELIMINARIES AND SETTING
  4. Notation.
  5. ${\ref{['def:quasar_convexity_constant']}}$-quasar convexity.
  6. $L$-smoothness.
  7. Problem setting.
  8. RELATED WORK
  9. THE ACCELERATED METHOD
  10. UNACCELERATED METHODS
  11. CONCLUSION
  12. PROPERTIES
  13. ACCELERATED CONSTRAINED METHOD FOR SMOOTH QUASAR-CONVEX OPTIMIZATION
  14. BINARY SEARCH
  15. Case 1, $h(a_k)>h(b_k)+\delta_1$
  16. ...and 4 more sections

Key Result

Lemma 1

[proof:lemma:properties_of_prox_and_moreau_env_smoothness] Let $f:\mathbb{R}^{d}\to\mathbb{R}$ have $L$-Lipschitz gradients on $\mathcal{X}$, and $\lambda = 1/(2L)$. For any $x \in \mathcal{X}$, the proximal problem $\min_{y\in\mathcal{X}} f(y) + \frac{1}{2\lambda}\| y-x \|_2^2$ is $L$-strongly conv

Figures (1)

  • Figure 1: \ref{['ex:quasar_convex_function']}; a $(1 / 2)$-quasar convex function $f(x) \stackrel{\mathrm{ def}}{=} (x^2+1/8)^{1 / 6}$ in $[-5, 5]$ (left) and its proximal subproblem $g(x) \stackrel{\mathrm{ def}}{=} f(x) + \frac{1}{2\lambda}(x-x_0)^2$, for $\lambda = 50$ and $x_0 \approx -12.23$ (right). The latter has a local maximizer at $x=-2$ and a local minimizer near $x=0$ and so it cannot be quasar convex in $[-5, 5]$.

Theorems & Definitions (22)

  • Lemma 1
  • Proposition 2
  • Lemma 3
  • Lemma 4
  • Theorem 5
  • Lemma 6
  • Lemma 7
  • Theorem 8: Projected Gradient Descent Rate
  • Theorem 9: Frank-Wolfe Rate
  • Remark 10
  • ...and 12 more