Deep Reinforcement Learning for Multi-Objective Optimization: Enhancing Wind Turbine Energy Generation while Mitigating Noise Emissions

TL;DR

This work tackles the problem of maximizing wind energy generation while mitigating acoustic emissions by deploying a Deep Reinforcement Learning controller for torque-pitch actuation. The approach uses a Double Deep Q-Network (DDQN) integrated with OpenFAST, Blade Element Momentum Theory, and the Brooks Pope–Marcolini aeroacoustic model to learn policies that balance $C_p$ and $\,\mathrm{SPL}$. A scalarized MORL reward is employed to locate policies near the Pareto front, with extensive validation on a SWT2.3-93 2.3 MW turbine under steady and turbulent winds; the Quiet DDQN can achieve substantial noise reduction with modest energy loss (about 22% under a 45 dB SPL cap). The methodology enables flexible, site-specific tuning of energy-noise objectives and demonstrates the potential of RL-based control to advance quieter, efficient wind energy systems.

Abstract

We develop a torque-pitch control framework using deep reinforcement learning for wind turbines to optimize the generation of wind turbine energy while minimizing operational noise. We employ a double deep Q-learning, coupled to a blade element momentum solver, to enable precise control over wind turbine parameters. In addition to the blade element momentum, we use the wind turbine acoustic model of Brooks Pope and Marcolini. Through training with simple winds, the agent learns optimal control policies that allow efficient control for complex turbulent winds. Our experiments demonstrate that the reinforcement learning is able to find optima at the Pareto front, when maximizing energy while minimizing noise. In addition, the adaptability of the reinforcement learning agent to changing turbulent wind conditions, underscores its efficacy for real-world applications. We validate the methodology using a SWT2.3-93 wind turbine with a rated power of 2.3 MW. We compare the reinforcement learning control to classic controls to show that they are comparable when not taking into account noise emissions. When including a maximum limit of 45 dB to the noise produced (100 meters downwind of the turbine), the extracted yearly energy decreases by 22%. The methodology is flexible and allows for easy tuning of the objectives and constraints through the reward definitions, resulting in a flexible multi-objective optimization framework for wind turbine control. Overall, our findings highlight the potential of RL-based control strategies to improve wind turbine efficiency while mitigating noise pollution, thus advancing sustainable energy generation technologies

Figures (13)

• Figure 1: Comparison of https://zenodo.org/records/7323750 dataset benchmarks and OpenFAST modeling of the SWT2.3-93 wind turbine.
• Figure 2: Sensitivity analyses for control parameters in relation to OSPL, power coefficient ($C_p$), and one-third octave SPL (dB A) spectra.
• Figure 3: Flow diagram of the reinforcement learning control methodology.
• Figure 4:
• Figure 6: Q-Network architecture.