Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerated Mirror Descent for Non-Euclidean Star-convex Functions

Clement Lezane, Sophie Langer, Wouter M Koolen

TL;DR

This work develops an accelerated optimization framework for weakly smooth star-convex functions in arbitrary normed spaces, extending beyond Euclidean geometry. By integrating a generalized binary search with accelerated mirror descent and a distance-generating function $\psi$ that is $(\mu,q)$-uniformly convex, the authors derive sharp convergence bounds for $F(x_{T+1}^{ag})-F_\star$, scalable with the norm regularity and Hölder smoothness parameter $\kappa$. The results show near-optimal rates for $p$-norms ($p>1$), and the methodology applies to composite norms, highlighting that norm regularity—not function convexity—drives acceleration in non-Euclidean settings. The paper also provides a lower-bound analysis for the binary-search component and discusses practical tuning, iteration complexity, and potential directions for constrained or stochastic extensions. These findings broaden the scope of accelerated methods to a wider class of nonconvex problems where non-Euclidean geometry is natural, with implications for high-dimensional optimization and learning in non-Euclidean spaces.

Abstract

Acceleration for non-convex functions is a fundamental challenge in optimisation. We revisit star-convex functions, which are strictly unimodal on all lines through a minimizer. [1] accelerate unconstrained star-convex minimization of functions that are smooth with respect to the Euclidean norm. To do so, they add a certain binary search step to gradient descent. In this paper, we accelerate unconstrained star-convex minimization of functions that are weakly smooth with respect to an arbitrary norm. We add a binary search step to mirror descent, generalize the approach and refine its complexity analysis. We prove that our algorithms have sharp convergence rates for star-convex functions with $α$-Holder continuous gradients and demonstrate that our rates are nearly optimal for $p$-norms. [1] Near-Optimal Methods for Minimizing Star-Convex Functions and Beyond, Hinder Oliver and Sidford Aaron and Sohoni Nimit

Accelerated Mirror Descent for Non-Euclidean Star-convex Functions

TL;DR

This work develops an accelerated optimization framework for weakly smooth star-convex functions in arbitrary normed spaces, extending beyond Euclidean geometry. By integrating a generalized binary search with accelerated mirror descent and a distance-generating function that is -uniformly convex, the authors derive sharp convergence bounds for , scalable with the norm regularity and Hölder smoothness parameter . The results show near-optimal rates for -norms (), and the methodology applies to composite norms, highlighting that norm regularity—not function convexity—drives acceleration in non-Euclidean settings. The paper also provides a lower-bound analysis for the binary-search component and discusses practical tuning, iteration complexity, and potential directions for constrained or stochastic extensions. These findings broaden the scope of accelerated methods to a wider class of nonconvex problems where non-Euclidean geometry is natural, with implications for high-dimensional optimization and learning in non-Euclidean spaces.

Abstract

Acceleration for non-convex functions is a fundamental challenge in optimisation. We revisit star-convex functions, which are strictly unimodal on all lines through a minimizer. [1] accelerate unconstrained star-convex minimization of functions that are smooth with respect to the Euclidean norm. To do so, they add a certain binary search step to gradient descent. In this paper, we accelerate unconstrained star-convex minimization of functions that are weakly smooth with respect to an arbitrary norm. We add a binary search step to mirror descent, generalize the approach and refine its complexity analysis. We prove that our algorithms have sharp convergence rates for star-convex functions with -Holder continuous gradients and demonstrate that our rates are nearly optimal for -norms. [1] Near-Optimal Methods for Minimizing Star-Convex Functions and Beyond, Hinder Oliver and Sidford Aaron and Sohoni Nimit
Paper Structure (28 sections, 25 theorems, 180 equations, 1 table, 2 algorithms)

This paper contains 28 sections, 25 theorems, 180 equations, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Literature review
  3. Our contributions
  4. Problem setting and preliminaries
  5. Definitions
  6. Main results
  7. Convergence result
  8. Proof Sketch of Theorem \ref{['thm-p>2']} (Convergence rate of Algorithm \ref{['algo-acc']})
  9. Binary search for Hölder smooth functions
  10. Polynomial growth of our iterates
  11. Lower bound for Binary search
  12. Summary
  13. Conclusion and future research
  14. A regularity property of norms
  15. Useful lemma for convergence proof
  16. ...and 13 more sections

Key Result

Theorem 3.1

In the setting of Assumption ass with $\kappa < q$, Algorithm algo-acc with the tuning below returns $x_{T+1}^{ag}$ after $T$ iterations with precision where $\mathcal{O}_{q, \kappa}$ omits constants depending only on $(q,\kappa)$. This is achieved for any $\alpha > 0$ by

Theorems & Definitions (47)

  • Definition 2.1: $\tau$-star-convexity
  • Definition 2.2: Bregman divergence
  • Definition 2.3: Weak smoothness
  • Definition 2.4: Uniform convexity
  • Example 3.1: $p$-norms
  • Example 3.2: Composite norms and distance generating function
  • Theorem 3.1
  • Corollary 3.2
  • Theorem 4.1
  • Theorem 4.2
  • ...and 37 more