Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Distributed Optimization for Large-Scale Quadratic Programming

Augustinos D. Saravanos, Hunter Kuperman, Alex Oshin, Arshiya Taj Abdul, Vincent Pacelli, Evangelos A. Theodorou

TL;DR

This work tackles the challenge of solving large-scale constrained quadratic programs by marrying distributed optimization with deep unfolding. It introduces DistributedQP, a convergence-guaranteed distributed solver that fuses OSQP with a consensus ADMM structure, and DeepDistributedQP, which unfolds DistributedQP into a learnable architecture that optimizes algorithmic parameters via learned feedback policies. The authors establish PAC-Bayes generalization bounds for unseen problems and demonstrate that DeepDistributedQP scales to problems with tens of thousands of variables and hundreds of thousands of constraints while achieving orders-of-magnitude faster wall-clock times than OSQP. They also show that training on small problems transfers to large-scale instances, and that local policies can outperform shared ones on structured problems. Overall, the paper provides a scalable, certifiable framework for learned distributed optimization with strong empirical performance and theoretical guarantees.

Abstract

Quadratic programming (QP) forms a crucial foundation in optimization, encompassing a broad spectrum of domains and serving as the basis for more advanced algorithms. Consequently, as the scale and complexity of modern applications continue to grow, the development of efficient and reliable QP algorithms is becoming increasingly vital. In this context, this paper introduces a novel deep learning-aided distributed optimization architecture designed for tackling large-scale QP problems. First, we combine the state-of-the-art Operator Splitting QP (OSQP) method with a consensus approach to derive DistributedQP, a new method tailored for network-structured problems, with convergence guarantees to optimality. Subsequently, we unfold this optimizer into a deep learning framework, leading to DeepDistributedQP, which leverages learned policies to accelerate reaching to desired accuracy within a restricted amount of iterations. Our approach is also theoretically grounded through Probably Approximately Correct (PAC)-Bayes theory, providing generalization bounds on the expected optimality gap for unseen problems. The proposed framework, as well as its centralized version DeepQP, significantly outperform their standard optimization counterparts on a variety of tasks such as randomly generated problems, optimal control, linear regression, transportation networks and others. Notably, DeepDistributedQP demonstrates strong generalization by training on small problems and scaling to solve much larger ones (up to 50K variables and 150K constraints) using the same policy. Moreover, it achieves orders-of-magnitude improvements in wall-clock time compared to OSQP. The certifiable performance guarantees of our approach are also demonstrated, ensuring higher-quality solutions over traditional optimizers.

Deep Distributed Optimization for Large-Scale Quadratic Programming

TL;DR

This work tackles the challenge of solving large-scale constrained quadratic programs by marrying distributed optimization with deep unfolding. It introduces DistributedQP, a convergence-guaranteed distributed solver that fuses OSQP with a consensus ADMM structure, and DeepDistributedQP, which unfolds DistributedQP into a learnable architecture that optimizes algorithmic parameters via learned feedback policies. The authors establish PAC-Bayes generalization bounds for unseen problems and demonstrate that DeepDistributedQP scales to problems with tens of thousands of variables and hundreds of thousands of constraints while achieving orders-of-magnitude faster wall-clock times than OSQP. They also show that training on small problems transfers to large-scale instances, and that local policies can outperform shared ones on structured problems. Overall, the paper provides a scalable, certifiable framework for learned distributed optimization with strong empirical performance and theoretical guarantees.

Abstract

Quadratic programming (QP) forms a crucial foundation in optimization, encompassing a broad spectrum of domains and serving as the basis for more advanced algorithms. Consequently, as the scale and complexity of modern applications continue to grow, the development of efficient and reliable QP algorithms is becoming increasingly vital. In this context, this paper introduces a novel deep learning-aided distributed optimization architecture designed for tackling large-scale QP problems. First, we combine the state-of-the-art Operator Splitting QP (OSQP) method with a consensus approach to derive DistributedQP, a new method tailored for network-structured problems, with convergence guarantees to optimality. Subsequently, we unfold this optimizer into a deep learning framework, leading to DeepDistributedQP, which leverages learned policies to accelerate reaching to desired accuracy within a restricted amount of iterations. Our approach is also theoretically grounded through Probably Approximately Correct (PAC)-Bayes theory, providing generalization bounds on the expected optimality gap for unseen problems. The proposed framework, as well as its centralized version DeepQP, significantly outperform their standard optimization counterparts on a variety of tasks such as randomly generated problems, optimal control, linear regression, transportation networks and others. Notably, DeepDistributedQP demonstrates strong generalization by training on small problems and scaling to solve much larger ones (up to 50K variables and 150K constraints) using the same policy. Moreover, it achieves orders-of-magnitude improvements in wall-clock time compared to OSQP. The certifiable performance guarantees of our approach are also demonstrated, ensuring higher-quality solutions over traditional optimizers.

Paper Structure

This paper contains 71 sections, 8 theorems, 145 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. Distributed optimization with ADMM.
  5. Learning-to-optimize for distributed optimization.
  6. Learning-to-optimize for (centralized) QP.
  7. Generalization guarantees for learning-to-optimize.
  8. Distributed Quadratic Programming
  9. Problem Formulation
  10. DistributedQP: The Underlying Optimization Algorithm
  11. Convergence Guarantees
  12. The DeepDistributedQP Architecture
  13. Main Architecture
  14. Architecture overview.
  15. Learning feedback policies.
  16. ...and 56 more sections

Key Result

Theorem 1

If Assumption assumption: eta holds and $\alpha^k \in [1,2)$, then the iterates ${\bm{w}}^k$ generated by the DistributedQP algorithm converge to the optimal solution ${\bm{w}}^*$ of problem (eq: distr qp problem), as $k \rightarrow \infty$.

Figures (7)

  • Figure 1: Wall-clock time comparison. DeepDistributedQP, DistributedQP (ours) and OSQP on large-scale QPs.
  • Figure 2: Example of consensus mapping ${\mathcal{G}}$ in problem (\ref{['eq: distr qp problem']}).
  • Figure 3: The DeepDistributedQP architecture. The proposed framework relies on unrolling the DistributedQP optimizer as a supervised deep learning framework. In particular, we interpret its iterations (\ref{['eq: DistrQP x update']})-(\ref{['eq: DistrQP y update']}) as sequential network layers and introduce learnable components ( orange blocks) to facilitate reaching the desired accuracy after a predefined number of allowed iterations.
  • Figure 4: DeepQP: The centralized version of DeepDistributedQP which boils down to unfolding the standard OSQP method.
  • Figure 5: Small-scale centralized comparison between DeepQP and OSQP. Across all tested problems, DeepQP consistently outperforms OSQP (same per-iteration complexity using the indirect method).
  • ...and 2 more figures

Theorems & Definitions (14)

  • Theorem 1: Convergence guarantees for DistributedQP
  • Theorem 2: Implicit Differentiation of Indirect Method
  • Theorem 3: Generalization bound for DeepDistributedQP
  • Lemma 1
  • proof
  • Definition 1: Polar cones
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • ...and 4 more