Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient reformulations of ReLU deep neural networks for surrogate modelling in power system optimisation

Yogesh Pipada Sunil Kumar, S. Ali Pourmousavi, Jon A. R. Liisberg, Julian Lesmos-Vinasco

TL;DR

This paper tackles the challenge of embedding ReLU DNN surrogates within power-system optimization without incurring prohibitive MIP complexity. It introduces a convexified ReLU DNN (cvxd) whose weights beyond the first layer are constrained to be non-negative, enabling an exact LP reformulation when the network output is minimised. Applied to an energy aggregator bidding problem in the Danish mFRR market, the cvxd LP reformulation demonstrates comparable solution quality to piecewise linearisation and MIP-based methods while delivering substantial computational speedups and robustness across architectures. Unlike penalty-based LP relaxations, the proposed approach avoids penalty tuning and shows consistent performance, though it trades off some expressiveness due to monotonic convex constraints. The results indicate that cvxd ReLU DNNs offer a scalable, reliable surrogate modelling pathway for a wide range of modern power-system optimization tasks, with potential extensions to generator costs, emissions, and chance-constrained formulations.

Abstract

The ongoing decarbonisation of power systems is driving an increasing reliance on distributed energy resources, which introduces complex and nonlinear interactions that are difficult to capture in conventional optimisation models. As a result, machine learning based surrogate modelling has emerged as a promising approach, but integrating machine learning models such as ReLU deep neural networks (DNNs) directly into optimisation often results in nonconvex and computationally intractable formulations. This paper proposes a linear programming (LP) reformulation for a class of convexified ReLU DNNs with non-negative weight matrices beyond the first layer, enabling a tight and tractable embedding of learned surrogate models in optimisation. We evaluate the method using a case study on learning the prosumer's responsiveness within an aggregator bidding problem in the Danish tertiary capacity market. The proposed reformulation is benchmarked against state-of-the-art alternatives, including piecewise linearisation (PWL), MIP-based embedding, and other LP relaxations. Across multiple neural network architectures and market scenarios, the convexified ReLU DNN achieves solution quality comparable to PWL and MIP-based reformulations while significantly improving computational performance and preserving model fidelity, unlike penalty-based reformulations. The results demonstrate that convexified ReLU DNNs offer a scalable and reliable methodology for integrating learned surrogate models in optimisation, with applicability to a wide range of emerging power system applications.

Efficient reformulations of ReLU deep neural networks for surrogate modelling in power system optimisation

TL;DR

This paper tackles the challenge of embedding ReLU DNN surrogates within power-system optimization without incurring prohibitive MIP complexity. It introduces a convexified ReLU DNN (cvxd) whose weights beyond the first layer are constrained to be non-negative, enabling an exact LP reformulation when the network output is minimised. Applied to an energy aggregator bidding problem in the Danish mFRR market, the cvxd LP reformulation demonstrates comparable solution quality to piecewise linearisation and MIP-based methods while delivering substantial computational speedups and robustness across architectures. Unlike penalty-based LP relaxations, the proposed approach avoids penalty tuning and shows consistent performance, though it trades off some expressiveness due to monotonic convex constraints. The results indicate that cvxd ReLU DNNs offer a scalable, reliable surrogate modelling pathway for a wide range of modern power-system optimization tasks, with potential extensions to generator costs, emissions, and chance-constrained formulations.

Abstract

The ongoing decarbonisation of power systems is driving an increasing reliance on distributed energy resources, which introduces complex and nonlinear interactions that are difficult to capture in conventional optimisation models. As a result, machine learning based surrogate modelling has emerged as a promising approach, but integrating machine learning models such as ReLU deep neural networks (DNNs) directly into optimisation often results in nonconvex and computationally intractable formulations. This paper proposes a linear programming (LP) reformulation for a class of convexified ReLU DNNs with non-negative weight matrices beyond the first layer, enabling a tight and tractable embedding of learned surrogate models in optimisation. We evaluate the method using a case study on learning the prosumer's responsiveness within an aggregator bidding problem in the Danish tertiary capacity market. The proposed reformulation is benchmarked against state-of-the-art alternatives, including piecewise linearisation (PWL), MIP-based embedding, and other LP relaxations. Across multiple neural network architectures and market scenarios, the convexified ReLU DNN achieves solution quality comparable to PWL and MIP-based reformulations while significantly improving computational performance and preserving model fidelity, unlike penalty-based reformulations. The results demonstrate that convexified ReLU DNNs offer a scalable and reliable methodology for integrating learned surrogate models in optimisation, with applicability to a wide range of emerging power system applications.
Paper Structure (20 sections, 1 theorem, 12 equations, 7 figures, 3 tables)

This paper contains 20 sections, 1 theorem, 12 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Motivation and related work
  3. Objectives and contributions
  4. ReLU DNN LP reformulations
  5. Proposed ReLU DNN reformulation
  6. SOTA ReLU DNN reformulations
  7. Case study: Energy aggregator bidding optimisation
  8. Optimisation model
  9. ReLU DNN training strategy
  10. Simulation study and results
  11. Optimisation parameters
  12. Tractaility, accuracy and solution quality analysis
  13. Computational performance
  14. Solution quality and surrogate model accuracy analysis
  15. ReLU DNN length and width sensitivity analysis
  16. ...and 5 more sections

Key Result

Theorem 2.1

Let us define functions $c:\mathbb{R}^{n_0}\rightarrow \mathbb{R}$ and $g:\mathbb{R}^{n_0} \rightarrow \mathbb{R}^{e}$, where $e \in \mathbb{N}$. Let us also consider the ReLU DNN $f$ as defined in Definition def: ReLU DNN with a single output, i.e., $n_{L+1} = 1$. Thus, we can define a cost minimis For some $k \in \mathbb{N}$ and $k \leq L$, if $\boldsymbol{W_{l}} \geq 0 \text{ (element-wise)} \,

Figures (7)

  • Figure 1: ReLU function feasible space for LP reformulations adapted from zhang2023learning
  • Figure 2: Optimisation problem parameters
  • Figure 3: CPU runtime histogram for all surrogate models across all simulated scenarios
  • Figure 4: RMSE between actual and surrogate model $\boldsymbol{\lambda^{\text{P}}}$ across different price scenarios
  • Figure 5: Optimisation results for one low price instance
  • ...and 2 more figures

Theorems & Definitions (5)

  • Definition 1
  • Definition 2
  • Theorem 2.1
  • proof
  • Definition 3