Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spatio-Temporal Graph Convolutional Neural Networks for Physics-Aware Grid Learning Algorithms

Tong Wu, Ignacio Losada Carreno, Anna Scaglione, Daniel Arnold

TL;DR

The effectiveness of the proposed architectures for spatio-temporal graph convolutional and recurrent neural networks whose structure is inspired by the physics of power systems is demonstrated in forecasting the power grid state and in finding a stochastic policy for foresighted voltage control using deep reinforcement learning.

Abstract

This paper proposes a model-free Volt-VAR control (VVC) algorithm via the spatio-temporal graph ConvNet-based deep reinforcement learning (STGCN-DRL) framework, whose goal is to control smart inverters in an unbalanced distribution system. We first identify the graph shift operator (GSO) based on the power flow equations. Then, we develop a spatio-temporal graph ConvNet (STGCN), testing both recurrent graph ConvNets (RGCN) and convolutional graph ConvNets (CGCN) architectures, aimed at capturing the spatiotemporal correlation of voltage phasors. The STGCN layer performs the feature extraction task for the policy function and the value function of the reinforcement learning architecture, and then we utilize the proximal policy optimization (PPO) to search the action spaces for an optimum policy function and to approximate an optimum value function. We further utilize the low-pass property of voltage graph signal to introduce an GCN architecture for the the policy whose input is a decimated state vector, i.e. a partial observation. Case studies on the unbalanced 123-bus systems validate the excellent performance of the proposed method in mitigating instabilities and maintaining nodal voltage profiles within a desirable range.

Spatio-Temporal Graph Convolutional Neural Networks for Physics-Aware Grid Learning Algorithms

TL;DR

The effectiveness of the proposed architectures for spatio-temporal graph convolutional and recurrent neural networks whose structure is inspired by the physics of power systems is demonstrated in forecasting the power grid state and in finding a stochastic policy for foresighted voltage control using deep reinforcement learning.

Abstract

This paper proposes a model-free Volt-VAR control (VVC) algorithm via the spatio-temporal graph ConvNet-based deep reinforcement learning (STGCN-DRL) framework, whose goal is to control smart inverters in an unbalanced distribution system. We first identify the graph shift operator (GSO) based on the power flow equations. Then, we develop a spatio-temporal graph ConvNet (STGCN), testing both recurrent graph ConvNets (RGCN) and convolutional graph ConvNets (CGCN) architectures, aimed at capturing the spatiotemporal correlation of voltage phasors. The STGCN layer performs the feature extraction task for the policy function and the value function of the reinforcement learning architecture, and then we utilize the proximal policy optimization (PPO) to search the action spaces for an optimum policy function and to approximate an optimum value function. We further utilize the low-pass property of voltage graph signal to introduce an GCN architecture for the the policy whose input is a decimated state vector, i.e. a partial observation. Case studies on the unbalanced 123-bus systems validate the excellent performance of the proposed method in mitigating instabilities and maintaining nodal voltage profiles within a desirable range.
Paper Structure (35 sections, 8 theorems, 63 equations, 3 figures, 4 tables, 1 algorithm)

This paper contains 35 sections, 8 theorems, 63 equations, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Related Works
  4. Graph Convolutional Neural Network
  5. Power System State Estimation and Forecasting
  6. Reinforcement Learning for Voltage Control
  7. Contributions and Organization
  8. A Brief Review of Graph Signal Processing
  9. Graph Convolutional Neural Networks Architectures for Power Grid Signals
  10. Real-Valued Grid Graph System Operator
  11. GCN and GRN
  12. GCN
  13. GRN
  14. GSO for Partial Observation
  15. GCN and GRN Applications
  16. ...and 20 more sections

Key Result

Lemma 1

With a first order expansion of the phase term the quadratic term $\bm{v}_n\bm{v}_{m}^H$ can be approximated as: where $\bm{\varphi}_n$ is re-centered by $\bm{\varphi}^\top_n = [{\varphi}_{n_a}, {\varphi}_{n_b}, {\varphi}_{n_c}] \triangleq [\angle{v}_{n_a}, \angle{v}_{n_b}+\frac{2\pi}{3}, \angle{v}_{n_c}-\frac{2\pi}{3}]$, and $\Gamma$ is expressed as: Replacing $m$ with $n$ in approx1 and appr

Figures (3)

  • Figure 1: (Left) Information flow of the power systems, (middle) the GRN structure achieved by the RNN and GCN blocks and (Right) the GCN structure.
  • Figure 2: An example of PSSE and PSSF by GCN for the IEEE 118-bus system.
  • Figure 3: (a) and (b) have 3 smart inverters and (c) has 6 smart inverters. (a) and (b) illustrate the learning (training) curves of the GCN-DRL and GRN-DRL algorithms for voltage magnitude regulations with full and partial observations, respectively.

Theorems & Definitions (11)

  • Lemma 1
  • Proof 1
  • Proposition 1
  • Proof 2
  • Lemma 2: ramakrishna2021gridgraph
  • Proof 3
  • Proposition 2
  • Corollary 1
  • Proposition 3
  • Proposition 4
  • ...and 1 more