Deep Reinforcement Learning for Voltage Control and Renewable Accommodation Using Spatial-Temporal Graph Information

TL;DR

This work tackles voltage fluctuations in RER-rich distribution networks by learning a control policy that balances voltage stability, renewable accommodation, and generation costs. It introduces MG-ASTGCN to extract multi-time-scale spatial-temporal graph information from the distribution network and feeds these features into a DRL controller solved via off-policy Deep Deterministic Policy Gradient (DDPG). The proposed framework demonstrates faster convergence and superior performance against optimization-based benchmarks and standard DRL baselines on IEEE 33/69/118-bus radial networks, while enhancing robustness to generator failures. The results provide practical guidance on tuning objective weights to achieve an effective balance between voltage control and renewable integration, and offer insights into the spatial-temporal correlations that drive DN dynamics.

Abstract

Renewable energy resources (RERs) have been increasingly integrated into distribution networks (DNs) for decarbonization. However, the variable nature of RERs introduces uncertainties to DNs, frequently resulting in voltage fluctuations that threaten system security and hamper the further adoption of RERs. To incentivize more RER penetration, we propose a deep reinforcement learning (DRL)-based strategy to dynamically balance the trade-off between voltage fluctuation control and renewable accommodation. To further extract multi-time-scale spatial-temporal (ST) graphical information of a DN, our strategy draws on a multi-grained attention-based spatial-temporal graph convolution network (MG-ASTGCN), consisting of ST attention mechanism and ST convolution to explore the node correlations in the spatial and temporal views. The continuous decision-making process of balancing such a trade-off can be modeled as a Markov decision process optimized by the deep deterministic policy gradient (DDPG) algorithm with the help of the derived ST information. We validate our strategy on the modified IEEE 33, 69, and 118-bus radial distribution systems, with outcomes significantly outperforming the optimization-based benchmarks. Simulations also reveal that our developed MG-ASTGCN can to a great extent accelerate the convergence speed of DDPG and improve its performance in stabilizing node voltage in an RER-rich DN. Moreover, our method improves the DN's robustness in the presence of generator failures.

Figures (15)

• Figure 1: The system model and the presented work.
• Figure 2: The undirected graph of a DN.
• Figure 3: The framework of the proposed MG-ASTGCN.
• Figure 4: The structure of one ST component.
• Figure 5: The detailed process of one recent graph segment passing through one whole ST component.