Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantum-Accelerated Deep Reinforcement Learning for Frequency Regulation Enhancement

Amin Masoumi, Mert Korkali

TL;DR

The paper tackles the sluggish adaptation of automatic generation control (AGC) due to static gains by introducing a quantum-accelerated deep reinforcement learning framework. It embeds parameterized quantum circuits (PQCs) into a Deep Deterministic Policy Gradient (DDPG) agent to enable adaptive, continuous control of grid frequency using near-term quantum devices. The authors design a hardware-efficient ansatz and integrate quantum updates into both the actor and critic networks, validating the approach on the IEEE 14-bus test system under a large load disturbance. Results indicate that the parameter-shift gradient with the PQC-based actor-critic yields improved convergence and frequency stability, while highlighting the practical considerations of noise and circuit depth on NISQ hardware. The work demonstrates a viable pathway for leveraging near-term quantum technology to enhance critical real-world control tasks in power systems.

Abstract

In modern power systems, frequency regulation is a fundamental prerequisite for ensuring system reliability and assessing the robustness of expansion projects. Conventional feedback control schemes, however, exhibit limited accuracy under varying operating conditions because their gains remain static. Consequently, deep reinforcement learning methods are increasingly employed to design adaptive controllers that can be generalized to diverse frequency control tasks. At the same time, recent advances in quantum computing provide avenues for embedding quantum capabilities into such critical applications. In particular, the potential of quantum algorithms can be more effectively explored and harnessed on near-term quantum devices by leveraging insights from active controller design. In this work, we incorporate a quantum circuit together with an ansatz into the operation of a deep deterministic policy gradient agent. The simulation results of the IEEE 14-bus test system demonstrate the potential of this integrated approach that can achieve reliable, robust performance across diverse real-world challenges.

Quantum-Accelerated Deep Reinforcement Learning for Frequency Regulation Enhancement

TL;DR

The paper tackles the sluggish adaptation of automatic generation control (AGC) due to static gains by introducing a quantum-accelerated deep reinforcement learning framework. It embeds parameterized quantum circuits (PQCs) into a Deep Deterministic Policy Gradient (DDPG) agent to enable adaptive, continuous control of grid frequency using near-term quantum devices. The authors design a hardware-efficient ansatz and integrate quantum updates into both the actor and critic networks, validating the approach on the IEEE 14-bus test system under a large load disturbance. Results indicate that the parameter-shift gradient with the PQC-based actor-critic yields improved convergence and frequency stability, while highlighting the practical considerations of noise and circuit depth on NISQ hardware. The work demonstrates a viable pathway for leveraging near-term quantum technology to enhance critical real-world control tasks in power systems.

Abstract

In modern power systems, frequency regulation is a fundamental prerequisite for ensuring system reliability and assessing the robustness of expansion projects. Conventional feedback control schemes, however, exhibit limited accuracy under varying operating conditions because their gains remain static. Consequently, deep reinforcement learning methods are increasingly employed to design adaptive controllers that can be generalized to diverse frequency control tasks. At the same time, recent advances in quantum computing provide avenues for embedding quantum capabilities into such critical applications. In particular, the potential of quantum algorithms can be more effectively explored and harnessed on near-term quantum devices by leveraging insights from active controller design. In this work, we incorporate a quantum circuit together with an ansatz into the operation of a deep deterministic policy gradient agent. The simulation results of the IEEE 14-bus test system demonstrate the potential of this integrated approach that can achieve reliable, robust performance across diverse real-world challenges.

Paper Structure

This paper contains 11 sections, 18 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Frequency Regulation via Automatic Generation Control
  3. Proposed Quantum-Accelerated DRL
  4. Fundamentals of DDPG
  5. Quantum Computing Preliminaries
  6. The Architecture of Quantum-Accelerated DDPG
  7. Quantum Integration of Actor Network
  8. Quantum Integration of Critic Network
  9. Case Study Description
  10. Results and Discussions
  11. Concluding Remarks

Figures (5)

  • Figure 1: The training process of the agent via the adjoint method.
  • Figure 2: The training process of the agent via the parameter-shift rule.
  • Figure 3: The training process without exploration noise
  • Figure 4: The training process quantum-accelerated agent
  • Figure 5: The training process of the agent via the parameter-shift rule.