Universal and Parameter-free Gradient Sliding for Composite Optimization

Abstract

We propose a parameter-free universal gradient sliding (PFUGS) algorithm for computing an approximation solution to the convex composite optimization problem $\min_{x\in X} \{f(x) + g(x)\}$. When $f$ and $g$ have $(M_ν,ν)$-Hölder and $L$-Lipschitz continuous (sub)gradients respectively, our proposed PFUGS method computes an approximate solution within at most $\mathcal{O}((M_ν/\varepsilon)^{{2}/{(1+3ν)}})$ and $\mathcal{O}((L/\varepsilon)^{1/2})$ evaluations of (sub)gradients of $f$ and $g$ respectively. Moreover, the PFUGS algorithm is parameter-free and does not require any prior knowledge on problem constants $ν$, $M_ν$, and $L$. To the best of knowledge, for problems involving two functions with different sets of problem constants, PFUGS is the first gradient sliding algorithm that is parameter-free.

Theorems & Definitions (44)

• Lemma 2.1
• Lemma 2.2
• Theorem 2.3
• proof
• Corollary 2.4
• proof
• Lemma 3.1
• Corollary 3.2
• Corollary 3.3
• Proposition 4.1