Wide-Area Feedback Control for Renewables-Heavy Power Systems: A Comparative Study of Reinforcement Learning and Lyapunov-Based Design

TL;DR

This work tackles the challenge of stabilizing renewables-heavy power systems by comparing a completely model-free wide-area damping control (WADC) design based on deep deterministic policy gradient ($DDPG$) with a model-based Lyapunov-based WADC computed via LMIs. The problem is formulated on detailed $NDAE$ power-system dynamics that include wind, solar, and composite loads, and is evaluated on IEEE 9-bus and 39-bus networks. Results show that the model-free approach can learn stabilizing policies without explicit system models but requires extensive training and lacks formal stability guarantees, while the model-based design provides rigorous stability proofs and faster deployment but depends on accurate system matrices. The study highlights the trade-offs between data-driven and physics-based control in future grids, suggesting paths toward robustness, sparsity, and scalability in wide-area control. Overall, the paper clarifies when each paradigm may be preferable and points to hybrid or robustness-enhanced methods for practical, renewables-rich power systems.

Abstract

As renewable energy sources become more prevalent, accurately modeling power grid dynamics is becoming increasingly more complex. Concurrently, data acquisition and realtime system state monitoring are becoming more available for control centers. This motivates shifting from \textit{model- and Lyapunov-based} feedback controller designs toward \textit{model-free} ones. Reinforcement learning (RL) has emerged as a key tool for designing model-free controllers. Various studies have been carried out to study voltage/frequency control strategies via RL. However, usually a simplified system model is used neglecting detailed dynamics of solar, wind, and composite loads -- and damping system-wide oscillations and modeling power flows are all usually ignored. To that end, we pose an optimal feedback control problem for a detailed renewables-heavy power system, defined by a set of nonlinear differential algebraic equations (NDAE). The control problem is solved using a completely model-free design via RL as well as using a model-based approach built upon the Lyapunov stability theory with guarantees. The paper in its essence seeks to explore whether data-driven feedback control should be used in power grids over its model-driven counterpart. Theoretical developments and thorough case studies are presented with an eye on this exploration. Finally, a detailed analysis is provided to delineate the strengths and weaknesses of both approaches for renewables-heavy grids.

Figures (10)

• Figure 1: Average and episodic rewards for 9-bus (left) and 39-bus (right) test systems.
• Figure 2: Comparative analysis under $\Delta_L = -0.5$ for 9-bus system: relative slip and angular speed of wind power plant (above), slip and rotor frequency of synchronous machine (below).
• Figure 3: Comparative analysis under $\Delta_L = 0.7$ for 9-bus system: DC link voltage of the wind and solar plant (above), relative speed of wind and solar plants (below).
• Figure 4: Comparative analysis under $\Delta_L = 0.003$ for 39-bus test system: relative slip and angular speed of all wind power plants (above), relative angle and reactive power output of wind plant connected at Bus 32 (below).
• Figure 5: Comparative analysis under $\Delta_L = -0.001$ and $\Delta_{I_s} = 0.1$ for 39-bus test system: rotor angle and real power output of Generator at Bus 38 (above), relative speed of all wind power plants and relative angle of wind plant connected at Bus 34 (below).

Theorems & Definitions (2)

• Proposition 1
• proof