A Neural Network Framework for Discovering Closed-form Solutions to Quadratic Programs with Linear Constraints

TL;DR

This work introduces a closed-form neural-network framework for solving quadratic programs with linear constraints by exploiting the piecewise-linear structure of mp-QP solutions. It uses a learning-via-discovery (LvD) procedure to derive all NN parameters analytically from problem coefficients, producing exact, feasible solutions within discovered critical regions and avoiding training data dependence. The architecture comprises a Shadow Price Model that fixes the per-region slope in the first layer and a linear Solution Model that reconstructs the primal/dual variables, with a mechanism to combine region slopes via an Incidence Matrix and Direction Vector. The approach is demonstrated on DC-OPF problems across multiple IEEE bus systems, achieving near-machine-precision KKT compliance and substantial speedups over traditional solvers, even under uncertain inputs. Overall, the method offers an explainable, scalable, and highly efficient alternative to black-box ML surrogates for parametric optimization problems, with clear avenues for extending to broader constraint types and integer variables.

Abstract

Deep neural networks (DNNs) have been used to model complex optimization problems in many applications, yet have difficulty guaranteeing solution optimality and feasibility, despite training on large datasets. Training a NN as a surrogate optimization solver amounts to estimating a global solution function that maps varying problem input parameters to the corresponding optimal solutions. Work in multiparametric programming (mp) has shown that solutions to quadratic programs (QP) are piece-wise linear functions of the parameters, and researchers have suggested leveraging this property to model mp-QP using NN with ReLU activation functions, which also exhibit piecewise linear behaviour. This paper proposes a NN modeling approach and learning algorithm that discovers the exact closed-form solution to QP with linear constraints, by analytically deriving NN model parameters directly from the problem coefficients without training. Whereas generic DNN cannot guarantee accuracy outside the training distribution, the closed-form NN model produces exact solutions for every discovered critical region of the solution function. To evaluate the closed-form NN model, it was applied to DC optimal power flow problems in electricity management. In terms of Karush-Kuhn-Tucker (KKT) optimality and feasibility of solutions, it outperformed a classically trained DNN and was competitive with, or outperformed, a commercial analytic solver (Gurobi) at far less computational cost. For a long-range energy planning problem, it was able to produce optimal and feasible solutions for millions of input parameters within seconds.

Figures (8)

• Figure 1: Sensitivity of $\boldsymbol{\mu}$ with respect to $\theta_0$
• Figure 2: The QP Solver Model Architecture
• Figure 3: Illustration of slope update rules on an example PWL function.
• Figure 4: $f_\mu(\boldsymbol{\theta})$ model
• Figure 5: Critical regions and connectivity graph for an mp-QP with two parameters (Adapted from pistikopoulos2020multi)

Theorems & Definitions (5)

• Theorem 1
• Lemma 2
• Theorem 3
• proof
• proof