A New Complexity Result for Strongly Convex Optimization with Locally $α$-H{ö}lder Continuous Gradients
Xiaojun Chen, C. T. Kelley, Lei Wang
TL;DR
The paper addresses the iteration complexity of gradient descent with a fixed stepsize for strongly convex objectives whose gradients are locally $\alpha$-Hölder continuous ($0<\alpha\le 1$). It proves a bound of $O(\log(\varepsilon^{-1})\varepsilon^{2\alpha-2})$ iterations to reach $\|u_k-u^*\|\le \varepsilon$, generalizing the classical $\alpha=1$ case. By interpreting gradient descent as an explicit Euler discretization of the gradient flow, it derives discretization-error estimates and provides both a basic and a refined complexity result under mild initial-point conditions. Numerical experiments illustrate the impact of the stepsize and the Hölder exponent on convergence and final accuracy, highlighting practical implications for step-size selection under non-Lipschitz gradients. The results deepen understanding of first-order methods for non-Lipschitz gradients and guide optimization in broader smoothness regimes.
Abstract
In this paper, we present a new complexity result for the gradient descent method with an appropriately fixed stepsize for minimizing a strongly convex function with locally $α$-H{ö}lder continuous gradients ($0 < α\leq 1$). The complexity bound for finding an approximate minimizer with a distance to the true minimizer less than $\varepsilon$ is $O(\log (\varepsilon^{-1}) \varepsilon^{2 α- 2})$, which extends the well-known complexity result for $α= 1$.