Coordinated Power Smoothing Control for Wind Storage Integrated System with Physics-informed Deep Reinforcement Learning

TL;DR

The paper tackles wind power variability by integrating wake effects and battery degradation costs into a coordinated power smoothing control framework for Wind Storage Integrated Systems (WSIS). It proposes a physics-informed, multi-agent deep reinforcement learning approach (PAMA-DDPG) with a bi-level architecture that separates wind-farm optimization from BESS smoothing, and embeds a power-fluctuation differential equation into the learning process via a Physics-informed Neural Network (PINN). Key contributions include reformulating PSC as a bi-level Markov decision process, designing upper- and lower-level agents with tailored state, action, and reward structures, and achieving faster convergence and better performance than non-PINN baselines. Empirical results on WindFarmSimulator show approximately an 11% increase in total profit and around 19% reduction in power fluctuation compared with a model-predictive control baseline, demonstrating improved economic efficiency and grid reliability. The framework offers a principled pathway to real-time, data-driven WSIS optimization with practical implications for offshore/onshore wind farms leveraging battery storage.

Abstract

The Wind Storage Integrated System with Power Smoothing Control (PSC) has emerged as a promising solution to ensure both efficient and reliable wind energy generation. However, existing PSC strategies overlook the intricate interplay and distinct control frequencies between batteries and wind turbines, and lack consideration of wake effect and battery degradation cost. In this paper, a novel coordinated control framework with hierarchical levels is devised to address these challenges effectively, which integrates the wake model and battery degradation model. In addition, after reformulating the problem as a Markov decision process, the multi-agent reinforcement learning method is introduced to overcome the bi-level characteristic of the problem. Moreover, a Physics-informed Neural Network-assisted Multi-agent Deep Deterministic Policy Gradient (PAMA-DDPG) algorithm is proposed to incorporate the power fluctuation differential equation and expedite the learning process. The effectiveness of the proposed methodology is evaluated through simulations conducted in four distinct scenarios using WindFarmSimulator (WFSim). The results demonstrate that the proposed algorithm facilitates approximately an 11% increase in total profit and a 19% decrease in power fluctuation compared to the traditional methods, thereby addressing the dual objectives of economic efficiency and grid-connected energy reliability.

Figures (9)

• Figure 1: The structure of the wind storage integrated system.
• Figure 2: The coordinated bi-level control framework.
• Figure 3: The workflow of the MA-DDPG.
• Figure 4: The wind turbine structural system.
• Figure 5: One-day wind speed profile for 4 testing scenarios.