Bundle methods with quadratic cuts for deterministic and stochastic strongly convex optimization problems

Abstract

We introduce two new methods for deterministic convex optimization problems: QCC (Quadratic Cuts for Convex optimization) and QB (Quadratic Bundle method). We prove the complexity of these methods for composite optimization problems which are the sum of a convex function $\tilde h$ and of a strongly convex function $\tilde f$ with parameter $μ$. These methods use as building blocks quadratic approximations of the strongly convex function $\tilde f$ where the quadratic terms are of form $\fracμ{2}\|\cdot-x_i\|^2$ for trial points $x_i$ computed along iterations (when $μ=0$ the building blocks are linear approximations). We extend the idea of using quadratic approximations to pieces of the objective for some multistage stochastic optimization problems which have strongly convex recourse functions that we approximate as a maximum of quadratic cuts. We call DASC (Dynamic Approximation for Strongly Convex optimzation) the corresponding optimization method. When the cuts are linear, the method boils down to the popular Stochastic Dual Dynamic Programming (SDDP) method. We provide conditions ensuring strong convexity of the recourse functions and prove the convergence of DASC. Numerical experiments illustrate the performance and correctness of DASC, with DASC being much quicker than SDDP for large values of the constants of strong convexity.

Theorems & Definitions (21)

• Lemma 2.1
• proof
• Lemma 2.2
• proof
• Remark 2.3
• Lemma 2.4
• proof
• Lemma 2.5
• proof
• Theorem 2.6: Complexity of QCC