Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Parameter-free proximal bundle methods with adaptive stepsizes for hybrid convex composite optimization problems

Renato D. C. Monteiro, Honghao Zhang

Abstract

This paper develops a parameter-free adaptive proximal bundle method with two important features: 1) adaptive choice of variable prox stepsizes that "closely fits" the instance under consideration; and 2) adaptive criterion for making the occurrence of serious steps easier. Computational experiments show that our method performs substantially fewer consecutive null steps (i.e., a shorter cycle) while maintaining the number of serious steps under control. As a result, our method performs significantly less number of iterations than its counterparts based on a constant prox stepsize choice and a non-adaptive cycle termination criterion. Moreover, our method is very robust relative to the user-provided initial stepsize.

Parameter-free proximal bundle methods with adaptive stepsizes for hybrid convex composite optimization problems

Abstract

This paper develops a parameter-free adaptive proximal bundle method with two important features: 1) adaptive choice of variable prox stepsizes that "closely fits" the instance under consideration; and 2) adaptive criterion for making the occurrence of serious steps easier. Computational experiments show that our method performs substantially fewer consecutive null steps (i.e., a shorter cycle) while maintaining the number of serious steps under control. As a result, our method performs significantly less number of iterations than its counterparts based on a constant prox stepsize choice and a non-adaptive cycle termination criterion. Moreover, our method is very robust relative to the user-provided initial stepsize.

Paper Structure

This paper contains 15 sections, 13 theorems, 78 equations, 5 tables.

Table of Contents

  1. Introduction
  2. Basic definitions and notation
  3. Main problem and algorithm
  4. Main problem
  5. Algorithm
  6. Bounding cycle lengths
  7. Bounding the number of cycles
  8. Case where $\phi_*$ is known
  9. General case where $\phi_*$ is unknown
  10. Computational experiments
  11. $l_1$ feasibility problem
  12. Ad-GPB* versus GPB
  13. Polyak type methods
  14. Lagrangian cut problem
  15. Concluding remarks

Key Result

Theorem 2.1

Define where Then, Ad-GPB finds a pair $(\hat{y}_k,\hat{n}_k) \in \mathrm{dom}\, \phi \times \mathbb{R}$ satisfying $\phi(\hat{y}_k) - \phi_* \le \phi(\hat{y}_k) - \hat{n}_k \le \varepsilon$ in at most $4\bar{K}(\varepsilon)$ cycles and iterations.

Theorems & Definitions (25)

  • Theorem 2.1
  • Lemma 3.1
  • proof
  • Lemma 3.2
  • proof
  • Lemma 3.3
  • proof
  • Lemma 3.4
  • proof
  • Proposition 3.5
  • ...and 15 more