Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Principled Data Augmentation for Learning to Solve Quadratic Programming Problems

Chendi Qian, Christopher Morris

TL;DR

This work tackles data scarcity in learning-to-optimize for linear and quadratic programs by introducing principled data augmentations that preserve optimality. It formulates optimization-aware affine transformations of the KKT system to generate diverse yet solvable LCQP instances and integrates them into supervised and contrastive learning pipelines for MPNNs. Empirical results show consistent improvements in data-scarce regimes, strong transfer to out-of-distribution and larger problems, and competitive or superior performance compared with standard graph augmentation baselines. The approach enables more robust, data-efficient neural solvers for convex optimization with practical impact on resource allocation, logistics, and model training pipelines that rely on LP/QP solutions.

Abstract

Linear and quadratic optimization are crucial in numerous real-world applications, ranging from training machine learning models to solving integer linear programs. Recently, learning-to-optimize methods (L2O) for linear (LPs) or quadratic programs (QPs) using message-passing graph neural networks (MPNNs) have gained traction, promising lightweight, data-driven proxies for solving such optimization problems. For example, they replace the costly computation of strong branching scores in branch-and-bound solvers, thereby reducing the need to solve many such optimization problems. However, robust L2O MPNNs remain challenging in data-scarce settings, especially when addressing complex optimization problems such as QPs. This work introduces a principled approach to data augmentation tailored for QPs via MPNNs. Our method leverages theoretically justified data augmentation techniques to generate diverse yet optimality-preserving instances. Furthermore, we integrate these augmentations into a self-supervised contrastive learning framework, thereby pretraining MPNNs for improved performance on L2O tasks. Extensive experiments demonstrate that our approach improves generalization in supervised scenarios and facilitates effective transfer learning to related optimization problems.

Principled Data Augmentation for Learning to Solve Quadratic Programming Problems

TL;DR

This work tackles data scarcity in learning-to-optimize for linear and quadratic programs by introducing principled data augmentations that preserve optimality. It formulates optimization-aware affine transformations of the KKT system to generate diverse yet solvable LCQP instances and integrates them into supervised and contrastive learning pipelines for MPNNs. Empirical results show consistent improvements in data-scarce regimes, strong transfer to out-of-distribution and larger problems, and competitive or superior performance compared with standard graph augmentation baselines. The approach enables more robust, data-efficient neural solvers for convex optimization with practical impact on resource allocation, logistics, and model training pipelines that rely on LP/QP solutions.

Abstract

Linear and quadratic optimization are crucial in numerous real-world applications, ranging from training machine learning models to solving integer linear programs. Recently, learning-to-optimize methods (L2O) for linear (LPs) or quadratic programs (QPs) using message-passing graph neural networks (MPNNs) have gained traction, promising lightweight, data-driven proxies for solving such optimization problems. For example, they replace the costly computation of strong branching scores in branch-and-bound solvers, thereby reducing the need to solve many such optimization problems. However, robust L2O MPNNs remain challenging in data-scarce settings, especially when addressing complex optimization problems such as QPs. This work introduces a principled approach to data augmentation tailored for QPs via MPNNs. Our method leverages theoretically justified data augmentation techniques to generate diverse yet optimality-preserving instances. Furthermore, we integrate these augmentations into a self-supervised contrastive learning framework, thereby pretraining MPNNs for improved performance on L2O tasks. Extensive experiments demonstrate that our approach improves generalization in supervised scenarios and facilitates effective transfer learning to related optimization problems.

Paper Structure

This paper contains 52 sections, 9 theorems, 35 equations, 2 figures, 15 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Present work
  3. Related work
  4. MPNN and L2O
  5. Graph data augmentation
  6. Graph contrastive learning
  7. Instance generation
  8. Background
  9. Notations
  10. LCQP
  11. MPNNs for LCQPs
  12. Principled data augmentation for LCQPs
  13. A framework for valid transformations
  14. Expressivity
  15. Validity
  16. ...and 37 more sections

Key Result

Proposition 2.1

Let $I \coloneqq \left( \bm{Q}, \bm{A}, \bm{b}, \bm{c} \right)$ be a LCQP instance with optimal primal-dual solution $\bm{x}^*, \bm{\lambda}^*$. Consider a transformation defined by $T(I) \coloneq (\bm{Q}, \bm{M}_{22} \bm{A}, \bm{M}_{22} \bm{b}, \bm{c})$. Then the transformed problem preserves the p

Figures (2)

  • Figure 1: Overview of our framework for principled data augmentation for quadratic optimization problems. Given a QP instance, we apply transformations (e.g., adding/removing/scaling variables or constraints; see \ref{['sec:tf_example']}) to generate new instances, thereby augmenting the training dataset. We then use standard supervised learning or contrastive learning to train an MPNN.
  • Figure 2: Illustration of our heuristics. (a). Smaller $\bm{a}^\intercal\bm{c}$ is more likely to be inactive. (b). Lower $b$ is more likely to be inactive.

Theorems & Definitions (19)

  • Proposition 2.1
  • Definition 2.2: Efficiently recoverable transformation
  • Definition 2.3: Solution-independent transformation
  • Proposition 2.4
  • Proposition 2.5
  • Proposition 2.6
  • Proposition C.1
  • Proposition C.2
  • Corollary C.2.1
  • Proposition C.3
  • ...and 9 more