Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Primal-Dual Bundle Methods for Linear Equality-Constrained Problems

Zhuoqing Zheng, Tao Liu, Xuyang Wu

TL;DR

Primal-dual bundle methods address linear equality-constrained convex optimization by enriching dual ascent and augmented Lagrangian updates with proximal bundle surrogates in both primal and dual steps. The framework yields BDA and BMM, which generalize the primal-dual gradient method, DA, linearized MM, and MM, while supporting a range of surrogate models (Polyak, cutting-plane, Polyak cutting-plane, two-cut). The authors provide convergence guarantees under standard assumptions and demonstrate empirically that BDA and BMM converge faster and with greater robustness to parameter settings than baseline methods. This approach offers a scalable, robust alternative for structured convex optimization problems with linear equality constraints.

Abstract

Dual ascent (DA) and the method of multipliers (MM) are fundamental methods for solving linear equality-constrained convex optimization problems, and their dual updates can be viewed as the minimization of a proximal linear surrogate function of the negative Lagrange dual and augmented Lagrange dual function, respectively. However, the proximal linear surrogate function may suffer from low approximation accuracy, which leads to slow convergence of DA and MM. To accelerate their convergence, we adapt the proximal bundle surrogate framework that can incorporate a list of more accurate surrogate functions, to both the primal and the dual updates of DA and MM, leading to a family of novel primal-dual bundle methods. Our methods generalize the primal-dual gradient method, DA, the linearized MM, and MM. Under standard assumptions that allow for a broad range of surrogate functions, we prove theoretical convergence guarantees for the proposed methods. Numerical experiments demonstrate that our methods converge not only faster, but also significantly more robust with respect to the parameters compared to the primal-dual gradient method, DA, the linearized MM, and MM.

Primal-Dual Bundle Methods for Linear Equality-Constrained Problems

TL;DR

Primal-dual bundle methods address linear equality-constrained convex optimization by enriching dual ascent and augmented Lagrangian updates with proximal bundle surrogates in both primal and dual steps. The framework yields BDA and BMM, which generalize the primal-dual gradient method, DA, linearized MM, and MM, while supporting a range of surrogate models (Polyak, cutting-plane, Polyak cutting-plane, two-cut). The authors provide convergence guarantees under standard assumptions and demonstrate empirically that BDA and BMM converge faster and with greater robustness to parameter settings than baseline methods. This approach offers a scalable, robust alternative for structured convex optimization problems with linear equality constraints.

Abstract

Dual ascent (DA) and the method of multipliers (MM) are fundamental methods for solving linear equality-constrained convex optimization problems, and their dual updates can be viewed as the minimization of a proximal linear surrogate function of the negative Lagrange dual and augmented Lagrange dual function, respectively. However, the proximal linear surrogate function may suffer from low approximation accuracy, which leads to slow convergence of DA and MM. To accelerate their convergence, we adapt the proximal bundle surrogate framework that can incorporate a list of more accurate surrogate functions, to both the primal and the dual updates of DA and MM, leading to a family of novel primal-dual bundle methods. Our methods generalize the primal-dual gradient method, DA, the linearized MM, and MM. Under standard assumptions that allow for a broad range of surrogate functions, we prove theoretical convergence guarantees for the proposed methods. Numerical experiments demonstrate that our methods converge not only faster, but also significantly more robust with respect to the parameters compared to the primal-dual gradient method, DA, the linearized MM, and MM.

Paper Structure

This paper contains 27 sections, 9 theorems, 158 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Algorithm Development
  3. Preliminaries: DA, MM, and the proximal bundle model
  4. DA and MM
  5. The proximal bundle model
  6. Algorithm
  7. Candidates of the primal model and the dual model
  8. Solving the subproblems in BDA and BMM
  9. Convergence Analysis
  10. Assumptions and assisting lemmas
  11. Convergence of BDA
  12. Convergence of BMM
  13. Numerical Experiment
  14. Conclusion
  15. Proof for assisting lemmas
  16. ...and 12 more sections

Key Result

Lemma 2.1

The Lagrange dual of problem eq:QP is where is the dual function. Letting $x_{\lambda} = \operatorname{\arg\;\min}_x h(x) +\frac{1}{2}x^TCx +d^Tx + \lambda^T\tilde{A}x$, it holds that Furthermore, if $\lambda^\star$ is optimal to eq:subproblem_dual, then the unique optimal solution of problem eq:simplified_primal_prob is $x_{\lambda^\star}$.

Figures (7)

  • Figure 1: Surrogate functions in the Polyak model \ref{['eq:pol_model']}, cutting-plane model \ref{['eq:cpm']} and the Polyak cutting-plane model \ref{['eq:pcpm']}
  • Figure 2: Convergence performance of BMM (convex)
  • Figure 3: Convergence performance of BMM-D (convex)
  • Figure 4: Convergence performance of BDA (strongly convex)
  • Figure 5: Convergence performance of BDA-D (strongly convex)
  • ...and 2 more figures

Theorems & Definitions (21)

  • Remark 1
  • Lemma 2.1
  • proof
  • Lemma 3.1
  • proof
  • Lemma 3.2
  • proof
  • Lemma 3.3
  • proof
  • Lemma 3.4
  • ...and 11 more