Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Efficient Unsupervised Framework for Convex Quadratic Programs via Deep Unrolling

Linxin Yang, Bingheng Li, Tian Ding, Jianghua Wu, Akang Wang, Yuyi Wang, Jiliang Tang, Ruoyu Sun, Xiaodong Luo

TL;DR

The paper tackles efficiently solving convex quadratic programs by extending matrix-free PDHG methods through an unsupervised, algorithm-unrolled neural network, PDQP-Net. By unrolling the PDQP iterations and incorporating learnable components, PDQP-Net can replicate the PDQP sequence and inherit its convergence guarantees while enabling faster warm-starts. A key contribution is the KKT-informed unsupervised loss, which combines primal feasibility, dual feasibility, and primal-dual optimality gaps without requiring labeled solutions. Empirically, PDQP-Net delivers high-quality primal-dual predictions and accelerates PDQP by up to 45% on diverse distributions, with strong improvements over GNN baselines and demonstrated generalization to out-of-distribution QPs. This work highlights the practical value of integrating optimization theory with deep unrolling to accelerate large-scale QP solving.

Abstract

Quadratic programs (QPs) arise in various domains such as machine learning, finance, and control. Recently, learning-enhanced primal-dual hybrid gradient (PDHG) methods have shown great potential in addressing large-scale linear programs; however, this approach has not been extended to QPs. In this work, we focus on unrolling "PDQP", a PDHG algorithm specialized for convex QPs. Specifically, we propose a neural network model called "PDQP-net" to learn optimal QP solutions. Theoretically, we demonstrate that a PDQP-net of polynomial size can align with the PDQP algorithm, returning optimal primal-dual solution pairs. We propose an unsupervised method that incorporates KKT conditions into the loss function. Unlike the standard learning-to-optimize framework that requires optimization solutions generated by solvers, our unsupervised method adjusts the network weights directly from the evaluation of the primal-dual gap. This method has two benefits over supervised learning: first, it helps generate better primal-dual gap since the primal-dual gap is in the objective function; second, it does not require solvers. We show that PDQP-net trained in this unsupervised manner can effectively approximate optimal QP solutions. Extensive numerical experiments confirm our findings, indicating that using PDQP-net predictions to warm-start PDQP can achieve up to 45% acceleration on QP instances. Moreover, it achieves 14% to 31% acceleration on out-of-distribution instances.

An Efficient Unsupervised Framework for Convex Quadratic Programs via Deep Unrolling

TL;DR

The paper tackles efficiently solving convex quadratic programs by extending matrix-free PDHG methods through an unsupervised, algorithm-unrolled neural network, PDQP-Net. By unrolling the PDQP iterations and incorporating learnable components, PDQP-Net can replicate the PDQP sequence and inherit its convergence guarantees while enabling faster warm-starts. A key contribution is the KKT-informed unsupervised loss, which combines primal feasibility, dual feasibility, and primal-dual optimality gaps without requiring labeled solutions. Empirically, PDQP-Net delivers high-quality primal-dual predictions and accelerates PDQP by up to 45% on diverse distributions, with strong improvements over GNN baselines and demonstrated generalization to out-of-distribution QPs. This work highlights the practical value of integrating optimization theory with deep unrolling to accelerate large-scale QP solving.

Abstract

Quadratic programs (QPs) arise in various domains such as machine learning, finance, and control. Recently, learning-enhanced primal-dual hybrid gradient (PDHG) methods have shown great potential in addressing large-scale linear programs; however, this approach has not been extended to QPs. In this work, we focus on unrolling "PDQP", a PDHG algorithm specialized for convex QPs. Specifically, we propose a neural network model called "PDQP-net" to learn optimal QP solutions. Theoretically, we demonstrate that a PDQP-net of polynomial size can align with the PDQP algorithm, returning optimal primal-dual solution pairs. We propose an unsupervised method that incorporates KKT conditions into the loss function. Unlike the standard learning-to-optimize framework that requires optimization solutions generated by solvers, our unsupervised method adjusts the network weights directly from the evaluation of the primal-dual gap. This method has two benefits over supervised learning: first, it helps generate better primal-dual gap since the primal-dual gap is in the objective function; second, it does not require solvers. We show that PDQP-net trained in this unsupervised manner can effectively approximate optimal QP solutions. Extensive numerical experiments confirm our findings, indicating that using PDQP-net predictions to warm-start PDQP can achieve up to 45% acceleration on QP instances. Moreover, it achieves 14% to 31% acceleration on out-of-distribution instances.

Paper Structure

This paper contains 26 sections, 4 theorems, 51 equations, 3 figures, 8 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Quadratic Programs
  4. The PDQP Algorithm
  5. KKT-informed PDQP Unrolling
  6. Unrolling the PDQP Algorithm
  7. Supervised Learning and Primal-Dual Gaps
  8. KKT-informed Unsupervised Learning
  9. Experiments
  10. Experiment Settings
  11. Supervised-learning VS Unsupervised-learning
  12. The Effect of Warm-starting PDQP
  13. Comparing PDQP-Net against GNNs
  14. Generalizing to QPs from Diverse Distributions
  15. Understanding Why the Unsupervised Approach Works
  16. ...and 11 more sections

Key Result

Theorem 3.1

Given any QP instance $\mathcal{M} \coloneqq (Q,A,b,c,l,u)$ and its corresponding primal-dual sequence $(x^k, y^k)_{k \leq K}$ generated by the PDQP algorithm within $K$ iterations, there exists a $K$-layer PDQP-Net with parameter assignment $\Theta_{PDQP}$ that can output the same iterative solutio

Figures (3)

  • Figure 1: Overview of the proposed unsupervised learning framework. The left panel illustrates the architecture of the PDQP-Net, which is based on the algorithm-unrolling approach, while the right panel presents the KKT-informed unsupervised learning scheme.
  • Figure 2: Plot of the primal-dual gap of points generated by randomly perturbing the optimal solution, using the $\ell_2$ distance to measure the deviations. Each dot represents a perturbed solution. The figure illustrates the relationship between the perturbation distance and the corresponding primal-dual gap on two QPLIB instances, demonstrating solutions with small distances could have large primal-dual gap.
  • Figure 3: Plot of the primal-dual gap of each iteration during the training process of the supervised and the unsupervised framework, using the $\ell_2$ norm to compute the gap. Each dot represents an iteration. The figure demonstrates that the unsupervised framework always achieves better primal-dual gap over the supervised one.

Theorems & Definitions (6)

  • Theorem 3.1
  • Proposition 3.1
  • Remark 3.1
  • Proposition 3.2
  • Theorem B.1
  • proof