Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Certifying Primal-Dual Optimization Proxies for Large-Scale Batch Economic Dispatch

Michael Klamkin, Mathieu Tanneau, Pascal Van Hentenryck

TL;DR

The work tackles the challenge that optimization proxies, while fast on average, can exhibit large worst-case gaps in large-scale batch problems such as economic dispatch. It introduces a hybrid solver that uses primal-dual proxies and a duality-gap certificate to guarantee performance within a user-specified tolerance $\epsilon$, falling back to a classical solver when needed. A joint primal-dual training regime with normalization and a hinge-like $\Gamma^{\epsilon}$ loss, along with sampling-without-replacement, yields self-certifying proxies that maintain feasibility and provide scalable speedups. Numerical experiments on large PEGASE-based economic-dispatch problems demonstrate speedups exceeding 1000x with worst-case optimality bounded by $\epsilon$, enabling trustworthy, high-throughput deployments in power-system planning and operation.

Abstract

Recent research has shown that optimization proxies can be trained to high fidelity, achieving average optimality gaps under 1% for large-scale problems. However, worst-case analyses show that there exist in-distribution queries that result in orders of magnitude higher optimality gap, making it difficult to trust the predictions in practice. This paper aims at striking a balance between classical solvers and optimization proxies in order to enable trustworthy deployments with interpretable speed-optimality tradeoffs based on a user-defined optimality threshold. To this end, the paper proposes a hybrid solver that leverages duality theory to efficiently bound the optimality gap of predictions, falling back to a classical solver for queries where optimality cannot be certified. To improve the achieved speedup of the hybrid solver, the paper proposes an alternative training procedure that combines the primal and dual proxy training. Experiments on large-scale transmission systems show that the hybrid solver is highly scalable. The proposed hybrid solver achieves speedups of over 1000x compared to a parallelized simplex-based solver while guaranteeing a maximum optimality gap of 2%.

Self-Certifying Primal-Dual Optimization Proxies for Large-Scale Batch Economic Dispatch

TL;DR

The work tackles the challenge that optimization proxies, while fast on average, can exhibit large worst-case gaps in large-scale batch problems such as economic dispatch. It introduces a hybrid solver that uses primal-dual proxies and a duality-gap certificate to guarantee performance within a user-specified tolerance , falling back to a classical solver when needed. A joint primal-dual training regime with normalization and a hinge-like loss, along with sampling-without-replacement, yields self-certifying proxies that maintain feasibility and provide scalable speedups. Numerical experiments on large PEGASE-based economic-dispatch problems demonstrate speedups exceeding 1000x with worst-case optimality bounded by , enabling trustworthy, high-throughput deployments in power-system planning and operation.

Abstract

Recent research has shown that optimization proxies can be trained to high fidelity, achieving average optimality gaps under 1% for large-scale problems. However, worst-case analyses show that there exist in-distribution queries that result in orders of magnitude higher optimality gap, making it difficult to trust the predictions in practice. This paper aims at striking a balance between classical solvers and optimization proxies in order to enable trustworthy deployments with interpretable speed-optimality tradeoffs based on a user-defined optimality threshold. To this end, the paper proposes a hybrid solver that leverages duality theory to efficiently bound the optimality gap of predictions, falling back to a classical solver for queries where optimality cannot be certified. To improve the achieved speedup of the hybrid solver, the paper proposes an alternative training procedure that combines the primal and dual proxy training. Experiments on large-scale transmission systems show that the hybrid solver is highly scalable. The proposed hybrid solver achieves speedups of over 1000x compared to a parallelized simplex-based solver while guaranteeing a maximum optimality gap of 2%.
Paper Structure (18 sections, 1 theorem, 15 equations, 1 figure, 2 tables, 2 algorithms)

This paper contains 18 sections, 1 theorem, 15 equations, 1 figure, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Optimization Proxies
  4. Neural Network Verification
  5. ML-accelerated Optimization
  6. Contributions and Outline
  7. Self-Certifying Proxies
  8. Linear Programming and Duality
  9. Hybrid Solver
  10. Joint Primal-Dual Training
  11. Normalization
  12. Targeting an optimality tolerance
  13. Sampling without replacement
  14. Numerical Experiments
  15. Economic Dispatch Formulation
  16. ...and 3 more sections

Key Result

Theorem 1

Let $\theta \in \Theta$ and $\epsilon \geq 0$, and let $(\tilde{\mathbf{x}}, \tilde{\mathbf{y}})$ be the solution returned by Algorithm algo:fallback with input $\theta, \epsilon$. The solution $(\tilde{\mathbf{x}}, \tilde{\mathbf{y}})$ is at most $\epsilon$-suboptimal, i.e.,

Figures (1)

  • Figure 1: Speedup of hybrid solver on 1354_pegase as a function of the optimality threshold $\epsilon$.

Theorems & Definitions (2)

  • Theorem 1
  • proof