Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differentiation Through Black-Box Quadratic Programming Solvers

Connor W. Magoon, Fengyu Yang, Noam Aigerman, Shahar Z. Kovalsky

TL;DR

The paper tackles differentiating the solution of Quadratic Programs (QPs) with respect to problem parameters in a solver-agnostic, modular fashion. It introduces dQP, a differentiable layer built on a local active-set analysis that reduces the problem to a single symmetric linear system, enabling closed-form derivative computation after a forward solve with any QP solver accessible via qpsolvers. The key contributions include a rigorous active-set based sensitivity framework, a locally equivalent equality-constrained reformulation, explicit derivative formulas, and an open-source PyTorch implementation compatible with 15+ solvers. Empirical results show strong robustness and scalability, especially for large-scale sparse QPs and bi-level geometry optimization, outperforming several existing differentiable QP methods on CPU. This approach significantly broadens the applicability of differentiable optimization in neural networks and bi-level optimization by decoupling optimization from differentiation and enabling solver flexibility.

Abstract

Differentiable optimization has attracted significant research interest, particularly for quadratic programming (QP). Existing approaches for differentiating the solution of a QP with respect to its defining parameters often rely on specific integrated solvers. This integration limits their applicability, including their use in neural network architectures and bi-level optimization tasks, restricting users to a narrow selection of solver choices. To address this limitation, we introduce dQP, a modular and solver-agnostic framework for plug-and-play differentiation of virtually any QP solver. A key insight we leverage to achieve modularity is that, once the active set of inequality constraints is known, both the solution and its derivative can be expressed using simplified linear systems that share the same matrix. This formulation fully decouples the computation of the QP solution from its differentiation. Building on this result, we provide a minimal-overhead, open-source implementation ( https://github.com/cwmagoon/dQP ) that seamlessly integrates with over 15 state-of-the-art solvers. Comprehensive benchmark experiments demonstrate dQP's robustness and scalability, particularly highlighting its advantages in large-scale sparse problems.

Differentiation Through Black-Box Quadratic Programming Solvers

TL;DR

The paper tackles differentiating the solution of Quadratic Programs (QPs) with respect to problem parameters in a solver-agnostic, modular fashion. It introduces dQP, a differentiable layer built on a local active-set analysis that reduces the problem to a single symmetric linear system, enabling closed-form derivative computation after a forward solve with any QP solver accessible via qpsolvers. The key contributions include a rigorous active-set based sensitivity framework, a locally equivalent equality-constrained reformulation, explicit derivative formulas, and an open-source PyTorch implementation compatible with 15+ solvers. Empirical results show strong robustness and scalability, especially for large-scale sparse QPs and bi-level geometry optimization, outperforming several existing differentiable QP methods on CPU. This approach significantly broadens the applicability of differentiable optimization in neural networks and bi-level optimization by decoupling optimization from differentiation and enabling solver flexibility.

Abstract

Differentiable optimization has attracted significant research interest, particularly for quadratic programming (QP). Existing approaches for differentiating the solution of a QP with respect to its defining parameters often rely on specific integrated solvers. This integration limits their applicability, including their use in neural network architectures and bi-level optimization tasks, restricting users to a narrow selection of solver choices. To address this limitation, we introduce dQP, a modular and solver-agnostic framework for plug-and-play differentiation of virtually any QP solver. A key insight we leverage to achieve modularity is that, once the active set of inequality constraints is known, both the solution and its derivative can be expressed using simplified linear systems that share the same matrix. This formulation fully decouples the computation of the QP solution from its differentiation. Building on this result, we provide a minimal-overhead, open-source implementation ( https://github.com/cwmagoon/dQP ) that seamlessly integrates with over 15 state-of-the-art solvers. Comprehensive benchmark experiments demonstrate dQP's robustness and scalability, particularly highlighting its advantages in large-scale sparse problems.
Paper Structure (28 sections, 2 theorems, 21 equations, 10 figures, 8 tables, 1 algorithm)

This paper contains 28 sections, 2 theorems, 21 equations, 10 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Approach
  4. Problem Setup: Differentiating QPs
  5. KKT Conditions and Sensitivity Analysis
  6. Optimality Conditions.
  7. Active Set and Complementary Slackness.
  8. Derivatives via Sensitivity Analysis.
  9. Extracting Derivatives from a QP Solver's Solution
  10. Locally Equivalent Equality-Constrained QP.
  11. Explicit Differentiation.
  12. Numerical Computation.
  13. Implementation.
  14. Experimental Results
  15. Modularity and Performance.
  16. ...and 13 more sections

Key Result

Theorem 3.1

The QP eqn:rQP is locally equivalent to the reduced equality-constrained QP eqn:QP and its solution $\zeta^*(\theta) = (z^*(\theta),\lambda^*(\theta),\mu^*(\theta))$ admits the explicit form Furthermore, the optimal point can be explicitly differentiated to obtain

Figures (10)

  • Figure 1: Comparison of differentiable QP methods for projection onto the probability simplex, evaluated by (a) total solve and differentiation time and (b) solution accuracy (duality gap). For moderately sized problems, our approach, using the Gurobi QP solver, outperforms existing methods on both metrics. As problem size increases, our method remains efficient, while others become intractable.
  • Figure 2: Illustration of active set differentiation. Left: a QP is shown by its quadratic level sets and polyhedral feasible set; the solution lies on a facet of the boundary; perturbations of the constraints lead to perturbations in the solution. Right: the perturbation of the solution remains the same when inactive constraints are eliminated.
  • Figure 3: (a) Active-set parameter space, coloring regions where the active set is constant. (b) Condition number of the full and reduced linear systems along a line in parameter space.
  • Figure 4: Accuracy versus total forward/backward solve for the Maros-Meszaros dataset maros1999repository. Each point represents a solved problem; point size illustrates dimension; problems solved solely by dQP are circled. The legend shows percentages of success rates; the solvers dQP used and their counts are PIQP 100, Gurobi 9, ProxQP 9, Clarabel 7, OSQP 2, MOSEK 1, QPALM 1.
  • Figure 5: (a) Planar embedding of a large-scale ant mesh (15k vertices). Zoom-ins show: (top) a non-injective harmonic map with edge overlaps highlighted in red; (bottom) dQP’s injective solution to the bi-level problem \ref{['eqn:geo']}. (b) Scalability: solver runtimes as mesh size increases for a synthetic problem.
  • ...and 5 more figures

Theorems & Definitions (3)

  • Theorem 3.1
  • Theorem B.1
  • proof