Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantum-Embedded Dynamic Security Control using Hybrid Deep Reinforcement Learning

Amin Masoumi, Mert Korkali

TL;DR

Inverter-based resource adoption erodes grid inertia, challenging transient stability and dynamic security control. The authors propose a quantum-embedded DRL (QDRL) framework that blends predictive DRL with parameterized quantum circuits to model the actor and critic in a DDPG setup, aiming to improve learning efficiency and proactive control. Evaluated on the IEEE 39-bus system with 50% IBRs, the quantum-augmented approach achieves comparable policy performance to vanilla DDPG while using far fewer trainable parameters, though it remains sensitive to NISQ-era noise. The study highlights both the potential of quantum-enhanced learning for DSC and the practical limitations that must be addressed for real-world deployment.

Abstract

Dynamic security control (DSC) is considered a pivotal step for the future power grid, which is increasingly penetrated by inverter-based resources. However, the efficiency of such practices, whether governed by automatic generation control or virtual inertia scheduling, can be intractable due to the complexity of the problem and the need to solve the differentialalgebraic equation in a timely manner with the required accuracy. In this regard, the model-free deep reinforcement learning algorithm demonstrates reliable performance. In addition, the introduction of fault-tolerant and near-term quantum computing terminologies, i.e., noisy intermediate-scale quantum, opens avenues for improving the performance of model-free algorithms leveraging quantum capabilities. This paper provides an organized framework and assesses its dependability by evaluating the performance of a quantum-embedded algorithm on the DSC of the IEEE 39-bus test system. Hence, the obtained results demonstrate promising applications, along with shortcomings that can be addressed and further developed later.

Quantum-Embedded Dynamic Security Control using Hybrid Deep Reinforcement Learning

TL;DR

Inverter-based resource adoption erodes grid inertia, challenging transient stability and dynamic security control. The authors propose a quantum-embedded DRL (QDRL) framework that blends predictive DRL with parameterized quantum circuits to model the actor and critic in a DDPG setup, aiming to improve learning efficiency and proactive control. Evaluated on the IEEE 39-bus system with 50% IBRs, the quantum-augmented approach achieves comparable policy performance to vanilla DDPG while using far fewer trainable parameters, though it remains sensitive to NISQ-era noise. The study highlights both the potential of quantum-enhanced learning for DSC and the practical limitations that must be addressed for real-world deployment.

Abstract

Dynamic security control (DSC) is considered a pivotal step for the future power grid, which is increasingly penetrated by inverter-based resources. However, the efficiency of such practices, whether governed by automatic generation control or virtual inertia scheduling, can be intractable due to the complexity of the problem and the need to solve the differentialalgebraic equation in a timely manner with the required accuracy. In this regard, the model-free deep reinforcement learning algorithm demonstrates reliable performance. In addition, the introduction of fault-tolerant and near-term quantum computing terminologies, i.e., noisy intermediate-scale quantum, opens avenues for improving the performance of model-free algorithms leveraging quantum capabilities. This paper provides an organized framework and assesses its dependability by evaluating the performance of a quantum-embedded algorithm on the DSC of the IEEE 39-bus test system. Hence, the obtained results demonstrate promising applications, along with shortcomings that can be addressed and further developed later.

Paper Structure

This paper contains 12 sections, 18 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction and Background
  2. Problem Formulation
  3. Dynamic Security Control via QDRL
  4. Preliminaries of DDPG
  5. Predictive DRL
  6. Preliminaries of Quantum Computing
  7. Application of PQC in DDPG
  8. Actor Network
  9. Critic Network
  10. Case Study Description and Implementation
  11. Results and Discussions
  12. Conclusion and Future Directions

Figures (4)

  • Figure 1: The building blocks of the proposed hybrid DRL framework.
  • Figure 2: The four-qubit ansatz for three observations and one action.
  • Figure 3: The reward function: the blue, green, and red curves indicate the quantum-embedded DDPG agent, vanilla DDPG agent, and the performance under noise. Also, the shaded areas show the standard deviation from the conducted simulation over 10 consecutive runs.
  • Figure 4: The rotor angle difference of the test system.