Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reinforcement Learning for Power-Flow Network Analysis

Alperen Ergur, Julia Lindberg, Vinny Miller

TL;DR

A probabilistic reward function that gives a good approximation to this root count, and a state-space that mimics the space of power flow equations are designed, which demonstrates the potential of RL for power-flow network design and analysis as well as the potential for RL to contribute meaningfully to problems that involve complex non-linear algebra or geometry.

Abstract

The power flow equations are non-linear multivariate equations that describe the relationship between power injections and bus voltages of electric power networks. Given a network topology, we are interested in finding network parameters with many equilibrium points. This corresponds to finding instances of the power flow equations with many real solutions. Current state-of-the art algorithms in computational algebra are not capable of answering this question for networks involving more than a small number of variables. To remedy this, we design a probabilistic reward function that gives a good approximation to this root count, and a state-space that mimics the space of power flow equations. We derive the average root count for a Gaussian model, and use this as a baseline for our RL agents. The agents discover instances of the power flow equations with many more solutions than the average baseline. This demonstrates the potential of RL for power-flow network design and analysis as well as the potential for RL to contribute meaningfully to problems that involve complex non-linear algebra or geometry. \footnote{Author order alphabetic, all authors contributed equally.

Reinforcement Learning for Power-Flow Network Analysis

TL;DR

A probabilistic reward function that gives a good approximation to this root count, and a state-space that mimics the space of power flow equations are designed, which demonstrates the potential of RL for power-flow network design and analysis as well as the potential for RL to contribute meaningfully to problems that involve complex non-linear algebra or geometry.

Abstract

The power flow equations are non-linear multivariate equations that describe the relationship between power injections and bus voltages of electric power networks. Given a network topology, we are interested in finding network parameters with many equilibrium points. This corresponds to finding instances of the power flow equations with many real solutions. Current state-of-the art algorithms in computational algebra are not capable of answering this question for networks involving more than a small number of variables. To remedy this, we design a probabilistic reward function that gives a good approximation to this root count, and a state-space that mimics the space of power flow equations. We derive the average root count for a Gaussian model, and use this as a baseline for our RL agents. The agents discover instances of the power flow equations with many more solutions than the average baseline. This demonstrates the potential of RL for power-flow network design and analysis as well as the potential for RL to contribute meaningfully to problems that involve complex non-linear algebra or geometry. \footnote{Author order alphabetic, all authors contributed equally.
Paper Structure (20 sections, 3 theorems, 83 equations, 4 figures, 5 tables)

This paper contains 20 sections, 3 theorems, 83 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Power Flow Networks and Their Non-Linear Equations
  3. Problem Statement and Related Challenges
  4. Our Contributions
  5. Related Work
  6. Machine Learning in Power Systems
  7. Counting Real Roots for Average-Case
  8. Stability Analysis via Root Finding
  9. Average Case Analysis
  10. Reward Function Design
  11. Reinforcement Learning Setup
  12. Problem formulation and Actor-Critic Architecture
  13. Reward Function Implementation
  14. Monte Carlo Implementation Details
  15. Experiments
  16. ...and 5 more sections

Key Result

Proposition 2.1

Figures (4)

  • Figure 1: $L=10, 15, 20$ agent test runs
  • Figure 2: Reward function comparison with $N=100000$, $M=2500$
  • Figure 3: Training plots for the $L=10, 15, 20$ matrix formulation agents.
  • Figure :

Theorems & Definitions (6)

  • Example 1.1
  • Proposition 2.1
  • proof
  • Theorem 2.2
  • Lemma 3.1
  • Remark 4.1