Model-Free Load Frequency Control of Nonlinear Power Systems Based on Deep Reinforcement Learning

TL;DR

A model-free LFC method for nonlinear power systems based on deep deterministic policy gradient framework is proposed, which can generate appropriate control actions and has strong adaptability for nonlinear power systems.

Abstract

Load frequency control (LFC) is widely employed in power systems to stabilize frequency fluctuation and guarantee power quality. However, most existing LFC methods rely on accurate power system modeling and usually ignore the nonlinear characteristics of the system, limiting controllers' performance. To solve these problems, this paper proposes a model-free LFC method for nonlinear power systems based on deep deterministic policy gradient (DDPG) framework. The proposed method establishes an emulator network to emulate power system dynamics. After defining the action-value function, the emulator network is applied for control actions evaluation instead of the critic network. Then the actor network controller is effectively optimized by estimating the policy gradient based on zeroth-order optimization (ZOO) and backpropagation algorithm. Simulation results and corresponding comparisons demonstrate the designed controller can generate appropriate control actions and has strong adaptability for nonlinear power systems.

Figures (12)

• Figure 1: Schematic diagram of the linearized LFC model.
• Figure 2: Nonlinear behaviors of the generator: GDB and GRC.
• Figure 3: Framework of model-free DDPG method.
• Figure 4: The calculation process of policy gradient.
• Figure 5: The rewards during training process on linearized power system.