Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data-driven Under Frequency Load Shedding Using Reinforcement Learning

Glory Justin, Santiago Paternain

TL;DR

The paper tackles under-frequency load shedding (UFLS) in modern, low-inertia power grids by formulating UFLS as a constrained Markov decision process and solving it with a Soft Actor-Critic (SAC) agent. To meet real-time requirements, it introduces data-driven frequency state assessment (FSA) classifiers—including a Graph Neural Network (GNN)—to replace computationally heavy dynamic simulations during training and operation. A dual objective is enforced via a CMDP, using a dual variable λ to trade off safety (adhering to frequency thresholds) and load shed, achieving high safety (up to ~92%) with minimal shedding when λ is tuned (e.g., λ = 20). The approach demonstrates robustness to partial observability, balanced shedding by prioritizing unsafe areas, and substantial reductions in training time and online computation, validated on the IEEE 68-bus system.

Abstract

Underfrequency load shedding (UFLS) is a critical control strategy in power systems aimed at maintaining system stability and preventing blackouts during severe frequency drops. Traditional UFLS schemes often rely on predefined rules and thresholds, which may not adapt effectively to the dynamic and complex nature of modern power grids. Reinforcement learning (RL) methods have been proposed to effectively handle the UFLS problem. However, training these RL agents is computationally burdensome due to solving multiple differential equations at each step of training. This computational burden also limits the effectiveness of the RL agents for use in real-time. To reduce the computational burden, a machine learning (ML) classifier is trained to capture the frequency response of the system to various disturbances. The RL agent is then trained using the classifier, thus avoiding multiple computations during each step of agent training. Key features of this approach include reduced training time, as well as faster real-time application compared to other RL agents, and its potential to improve system resilience by minimizing the amount of load shed while effectively stabilizing the frequency. Comparative studies with conventional UFLS schemes demonstrate that the RL-based strategy achieves superior performance while significantly reducing the time required. Simulation results on the IEEE 68-bus system validate the performance of the proposed RL method.

Data-driven Under Frequency Load Shedding Using Reinforcement Learning

TL;DR

The paper tackles under-frequency load shedding (UFLS) in modern, low-inertia power grids by formulating UFLS as a constrained Markov decision process and solving it with a Soft Actor-Critic (SAC) agent. To meet real-time requirements, it introduces data-driven frequency state assessment (FSA) classifiers—including a Graph Neural Network (GNN)—to replace computationally heavy dynamic simulations during training and operation. A dual objective is enforced via a CMDP, using a dual variable λ to trade off safety (adhering to frequency thresholds) and load shed, achieving high safety (up to ~92%) with minimal shedding when λ is tuned (e.g., λ = 20). The approach demonstrates robustness to partial observability, balanced shedding by prioritizing unsafe areas, and substantial reductions in training time and online computation, validated on the IEEE 68-bus system.

Abstract

Underfrequency load shedding (UFLS) is a critical control strategy in power systems aimed at maintaining system stability and preventing blackouts during severe frequency drops. Traditional UFLS schemes often rely on predefined rules and thresholds, which may not adapt effectively to the dynamic and complex nature of modern power grids. Reinforcement learning (RL) methods have been proposed to effectively handle the UFLS problem. However, training these RL agents is computationally burdensome due to solving multiple differential equations at each step of training. This computational burden also limits the effectiveness of the RL agents for use in real-time. To reduce the computational burden, a machine learning (ML) classifier is trained to capture the frequency response of the system to various disturbances. The RL agent is then trained using the classifier, thus avoiding multiple computations during each step of agent training. Key features of this approach include reduced training time, as well as faster real-time application compared to other RL agents, and its potential to improve system resilience by minimizing the amount of load shed while effectively stabilizing the frequency. Comparative studies with conventional UFLS schemes demonstrate that the RL-based strategy achieves superior performance while significantly reducing the time required. Simulation results on the IEEE 68-bus system validate the performance of the proposed RL method.
Paper Structure (15 sections, 25 equations, 5 figures, 4 tables, 1 algorithm)

This paper contains 15 sections, 25 equations, 5 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Main Contributions
  4. Problem Description
  5. RL for UFLS
  6. Modeling UFLS as a Constrained MDP.
  7. Adapted SAC Algorithm
  8. FSA Using ML
  9. Numerical Experiments
  10. FSA results
  11. UFLS in Full Observability
  12. Computational Efficiency
  13. UFLS in Partial Observability
  14. Balanced Load-Shedding
  15. Conclusion

Figures (5)

  • Figure 1: Frequency response after a disturbance at one of the generators in the IEEE 68-bus system. The disturbance here is a sudden increase in load $\Delta P$=2 MW at time $t=5$s. The frequency immediately drops following the disturbance and fails to recover to pre-contingency nominal frequency of 60Hz.
  • Figure 2: Illustration of the proposed UFLS scheme using RL. First, FSA is performed. If unsafe, UFLS is triggered. The agent outputs a set of load-shed coefficients. FSA is then repeated on the new state with the new load profile. If still unsafe, UFLS is repeated until safe.
  • Figure 3: Neural network approach to FSA compared to the conventional method.
  • Figure 4: Safety percentage achieved by RL agent on 100 test data points during training. Safety progressively increases with increasing $\lambda$ with $\lambda=20$ giving the maximum safety percentage.
  • Figure 5: Total power shed for 5 values of $\lambda$ using the GNN for FSA. $\lambda=20$ shows the highest load shed for the GNN while $\lambda=0$ generates the least amount of load shed.