Probabilistic Reachability Analysis of Multi-scale Voltage Dynamics Using Reinforcement Learning

TL;DR

A deep reinforcement learning-based framework for probabilistic reachability analysis of multi-scale voltage dynamics and enables consistent learning of risk probabilities associated with multiple instability types within a unified framework is presented.

Abstract

Voltage stability in modern power systems involves coupled dynamics across multiple time scales. Conventional methods based on time-scale separation or static stability margins may overlook instabilities caused by the coupling of slow and fast transients. Uncertainty in operating conditions further complicates stability assessment, and high computational cost of Monte Carlo simulations limit its applicability to multi-scale dynamics. This paper presents a deep reinforcement learning-based framework for probabilistic reachability analysis of multi-scale voltage dynamics. By formulating each instability mechanism as a distinct absorbing state and introducing a multi-critic architecture for mechanism-specific learning, the proposed method enables consistent learning of risk probabilities associated with multiple instability types within a unified framework. The approach is demonstrated on a four-bus system with load tap changers and over-excitation limiters, illustrating effectiveness of the proposed learning-based reachability analysis in identifying and quantifying the mechanisms leading to voltage collapse.

Figures (7)

• Figure 1: Loss of short-term equilibrium due to slow dynamics (modified from IEEE2002PES_TR9)
• Figure 2: One-line diagram of the example system VanCutsem1998
• Figure 3: Time evolution of an S-LT1 type generator instability
• Figure 4: Without corrective action
• Figure 5: With corrective action