WFCRL: A Multi-Agent Reinforcement Learning Benchmark for Wind Farm Control

TL;DR

The paper addresses the challenge of optimizing wind-farm power production under wake interactions using reinforcement learning. It introduces WFCRL, the first open, highly configurable MARL benchmark for wind-farm control, interfaced with two simulators: FLORIS for static, low-fidelity modeling and FAST.Farm for dynamic, higher-fidelity modeling. Key contributions include a MARL framework with decentralized agents, a suite of 10 wind layouts per simulator (including real farms), three wind-condition scenarios, and implementations of IPPO, MAPPO, QMIX, and IDQN/IDRQN baselines, plus a wake-steering benchmark that accounts for fatigue costs. Experiments reveal method- and scenario-dependent performance, demonstrate transfer-learning potential between simulators, and highlight challenges in bridging static-trained policies to dynamic real-world dynamics. Overall, WFCRL provides reproducible, extensible tools to evaluate cooperative MARL approaches for wind energy, aiming to connect RL research with practical wind-farm control applications.

Abstract

The wind farm control problem is challenging, since conventional model-based control strategies require tractable models of complex aerodynamical interactions between the turbines and suffer from the curse of dimension when the number of turbines increases. Recently, model-free and multi-agent reinforcement learning approaches have been used to address this challenge. In this article, we introduce WFCRL (Wind Farm Control with Reinforcement Learning), the first open suite of multi-agent reinforcement learning environments for the wind farm control problem. WFCRL frames a cooperative Multi-Agent Reinforcement Learning (MARL) problem: each turbine is an agent and can learn to adjust its yaw, pitch or torque to maximize the common objective (e.g. the total power production of the farm). WFCRL also offers turbine load observations that will allow to optimize the farm performance while limiting turbine structural damages. Interfaces with two state-of-the-art farm simulators are implemented in WFCRL: a static simulator (FLORIS) and a dynamic simulator (FAST.Farm). For each simulator, $10$ wind layouts are provided, including $5$ real wind farms. Two state-of-the-art online MARL algorithms are implemented to illustrate the scaling challenges. As learning online on FAST.Farm is highly time-consuming, WFCRL offers the possibility of designing transfer learning strategies from FLORIS to FAST.Farm.

Figures (12)

• Figure 1: Left: Wake effects in the offshore wind farm of Horns Rev 1 - Vattenfall. Right: Schema of a wind turbine boersma2017tutorial. The pitch, yaw or torque can be controlled.
• Figure 2: The evolution of total reward (b), power output (c) and load penalties (d) accumulated over an episode (with $T$=150) on the Ablaincourt environment, simulated with FLORIS. A visual representation of the layout is in (a), where the coordinates are in wind turbine diameters. During training, policies are evaluated every $5$ training steps with deterministic policies. The curves are plotted for all $5$ seeds.
• Figure 3: Evolution of the evaluation score, defined in \ref{['eq:eval_eq']}, during the training of IPPO and MAPPO on the two environments Turb3Row1 (left) and Ablaincourt (right).
• Figure 4: Wind velocity field for the simulation of our 3-turbines layout on the 2 simulators: FLORIS and FAST.Farm.
• Figure 5: Schema: interfacing infrastructure between FAST.Farm and Python