Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Blackout Mitigation via Physics-guided RL

Anmol Dwivedi, Santiago Paternain, Ali Tajer

TL;DR

The paper tackles preventing blackouts in stressed power grids by learning sequences of real-time remedial actions that combine line switching and generator adjustments. It introduces a physics-guided RL framework that uses sensitivity factors (PTDF and LODF) to structure exploration and improve policy learning, implemented with a dueling DQN and prioritized experience replay. Empirical results on Grid2Op using 36-bus and IEEE 118-bus networks show that physics-guided exploration yields longer survival times, with strategic line removals and coordinated generator dispatch outperforming purely data-driven or baseline approaches. The work demonstrates the practical value of integrating physical network signals into RL for resilient grid operation and outlines scalability considerations and broader applicability.

Abstract

This paper considers the sequential design of remedial control actions in response to system anomalies for the ultimate objective of preventing blackouts. A physics-guided reinforcement learning (RL) framework is designed to identify effective sequences of real-time remedial look-ahead decisions accounting for the long-term impact on the system's stability. The paper considers a space of control actions that involve both discrete-valued transmission line-switching decisions (line reconnections and removals) and continuous-valued generator adjustments. To identify an effective blackout mitigation policy, a physics-guided approach is designed that uses power-flow sensitivity factors associated with the power transmission network to guide the RL exploration during agent training. Comprehensive empirical evaluations using the open-source Grid2Op platform demonstrate the notable advantages of incorporating physical signals into RL decisions, establishing the gains of the proposed physics-guided approach compared to its black box counterparts. One important observation is that strategically~\emph{removing} transmission lines, in conjunction with multiple real-time generator adjustments, often renders effective long-term decisions that are likely to prevent or delay blackouts.

Blackout Mitigation via Physics-guided RL

TL;DR

The paper tackles preventing blackouts in stressed power grids by learning sequences of real-time remedial actions that combine line switching and generator adjustments. It introduces a physics-guided RL framework that uses sensitivity factors (PTDF and LODF) to structure exploration and improve policy learning, implemented with a dueling DQN and prioritized experience replay. Empirical results on Grid2Op using 36-bus and IEEE 118-bus networks show that physics-guided exploration yields longer survival times, with strategic line removals and coordinated generator dispatch outperforming purely data-driven or baseline approaches. The work demonstrates the practical value of integrating physical network signals into RL for resilient grid operation and outlines scalability considerations and broader applicability.

Abstract

This paper considers the sequential design of remedial control actions in response to system anomalies for the ultimate objective of preventing blackouts. A physics-guided reinforcement learning (RL) framework is designed to identify effective sequences of real-time remedial look-ahead decisions accounting for the long-term impact on the system's stability. The paper considers a space of control actions that involve both discrete-valued transmission line-switching decisions (line reconnections and removals) and continuous-valued generator adjustments. To identify an effective blackout mitigation policy, a physics-guided approach is designed that uses power-flow sensitivity factors associated with the power transmission network to guide the RL exploration during agent training. Comprehensive empirical evaluations using the open-source Grid2Op platform demonstrate the notable advantages of incorporating physical signals into RL decisions, establishing the gains of the proposed physics-guided approach compared to its black box counterparts. One important observation is that strategically~\emph{removing} transmission lines, in conjunction with multiple real-time generator adjustments, often renders effective long-term decisions that are likely to prevent or delay blackouts.
Paper Structure (38 sections, 22 equations, 1 figure, 7 tables, 4 algorithms)

This paper contains 38 sections, 22 equations, 1 figure, 7 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. System Model
  4. Blackout Mitigation Model
  5. Line-Switching Actions
  6. Generator Adjustments
  7. Maximizing Survival Time
  8. Blackout Mitigation as an MDP
  9. Modeling Failure Mitigation as an MDP
  10. MDP State Space $\mathcal{S}$
  11. Action Space $\mathcal{A}$
  12. Stochastic Transition Dynamics ${\mathbb {P}}$
  13. Reward Dynamics $\mathcal{R}$
  14. Physics-guided RL Solution
  15. Motivation
  16. ...and 23 more sections

Figures (1)

  • Figure 1: Agent$-$MDP interactions for both the Grid2Op 36-bus and the IEEE-118 bus systems with $\mathcal{A} = \mathcal{A}_{\sf line}$ and $\mu_{\sf line}=\mu_{\sf gen}=0$.