Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Coupling Smoothed Particle Hydrodynamics with Multi-Agent Deep Reinforcement Learning for Cooperative Control of Point Absorbers

Yi Zhan, Iván Martínez-Estévez, Min Luo, Alejandro J. C. Crespo, Abbas Khayyer

TL;DR

This work tackles the challenge of maximizing energy capture in wave energy converter arrays by linking a high-fidelity SPH fluid solver with cooperative multi-agent deep reinforcement learning. It introduces a GPU-accelerated, two-way SPH–MADRL framework using CTDE and the MASAC algorithm to learn adaptive PTO damping policies for multiple point absorbers. Validation against experiments confirms accurate fluid–structure interactions, and 2-D/3-D simulations demonstrate that learned policies substantially improve total energy absorption over fixed damping, with gains up to ~23% in 2-D and ~21% in 3-D irregular waves. The results show strong potential for scalable, real-time intelligent control in complex fluid–structure systems and lay the groundwork for broader applications in ocean engineering; the authors also commit to releasing open-source software for reproducibility and further development.

Abstract

Wave Energy Converters, particularly point absorbers, have emerged as one of the most promising technologies for harvesting ocean wave energy. Nevertheless, achieving high conversion efficiency remains challenging due to the inherently complex and nonlinear interactions between incident waves and device motion dynamics. This study develops an optimal adaptive damping control model for the power take-off (PTO) system by coupling Smoothed Particle Hydrodynamics (SPH) with multi-agent deep reinforcement learning. The proposed framework enables real-time communication between high-fidelity SPH simulations and intelligent control agents that learn coordinated policies to maximise energy capture. In each training episode, the SPH-based environment provides instantaneous hydrodynamic states to the agents, which output continuous damping actions and receive rewards reflecting power absorption. The Multi-Agent Soft Actor Critic algorithm is employed within a centralised-training and decentralised-execution scheme to ensure stable learning in continuous, multi-body systems. The entire platform is implemented in a unified GPU-accelerated C++ environment, allowing long-horizon training and large-scale three-dimensional simulations. The approach is validated through a series of two-dimensional and three-dimensional benchmark cases under regular and irregular wave conditions. Compared with constant PTO damping, the learned control policy increases overall energy capture by 23.8% and 21.5%, respectively, demonstrating the strong potential of intelligent control for improving the performance of wave energy converter arrays. The developed three-dimensional GPU-accelerated multi-agent platform in computational hydrodynamics, is extendable to other fluid-structure interaction engineering problem that require real-time, multi-body coordinated control.

Coupling Smoothed Particle Hydrodynamics with Multi-Agent Deep Reinforcement Learning for Cooperative Control of Point Absorbers

TL;DR

This work tackles the challenge of maximizing energy capture in wave energy converter arrays by linking a high-fidelity SPH fluid solver with cooperative multi-agent deep reinforcement learning. It introduces a GPU-accelerated, two-way SPH–MADRL framework using CTDE and the MASAC algorithm to learn adaptive PTO damping policies for multiple point absorbers. Validation against experiments confirms accurate fluid–structure interactions, and 2-D/3-D simulations demonstrate that learned policies substantially improve total energy absorption over fixed damping, with gains up to ~23% in 2-D and ~21% in 3-D irregular waves. The results show strong potential for scalable, real-time intelligent control in complex fluid–structure systems and lay the groundwork for broader applications in ocean engineering; the authors also commit to releasing open-source software for reproducibility and further development.

Abstract

Wave Energy Converters, particularly point absorbers, have emerged as one of the most promising technologies for harvesting ocean wave energy. Nevertheless, achieving high conversion efficiency remains challenging due to the inherently complex and nonlinear interactions between incident waves and device motion dynamics. This study develops an optimal adaptive damping control model for the power take-off (PTO) system by coupling Smoothed Particle Hydrodynamics (SPH) with multi-agent deep reinforcement learning. The proposed framework enables real-time communication between high-fidelity SPH simulations and intelligent control agents that learn coordinated policies to maximise energy capture. In each training episode, the SPH-based environment provides instantaneous hydrodynamic states to the agents, which output continuous damping actions and receive rewards reflecting power absorption. The Multi-Agent Soft Actor Critic algorithm is employed within a centralised-training and decentralised-execution scheme to ensure stable learning in continuous, multi-body systems. The entire platform is implemented in a unified GPU-accelerated C++ environment, allowing long-horizon training and large-scale three-dimensional simulations. The approach is validated through a series of two-dimensional and three-dimensional benchmark cases under regular and irregular wave conditions. Compared with constant PTO damping, the learned control policy increases overall energy capture by 23.8% and 21.5%, respectively, demonstrating the strong potential of intelligent control for improving the performance of wave energy converter arrays. The developed three-dimensional GPU-accelerated multi-agent platform in computational hydrodynamics, is extendable to other fluid-structure interaction engineering problem that require real-time, multi-body coordinated control.
Paper Structure (21 sections, 35 equations, 17 figures, 6 tables, 1 algorithm)

This paper contains 21 sections, 35 equations, 17 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Numerical methods
  3. SPH model
  4. Governing equations for the fluid
  5. Boundary condition
  6. Wave generation and absorption
  7. Wave-driven floating object
  8. Time-integration
  9. Deep reinforcement learning (DRL) model
  10. Soft Actor-Critic (SAC) algorithm
  11. Multi-agent deep reinforcement learning (MADRL)
  12. Two-way coupling of SPH and DRL
  13. Numerical investigations
  14. Validation of the SPH solver
  15. Wave generation and propagation
  16. ...and 6 more sections

Figures (17)

  • Figure 1: Coupling workflow between the SPH solver and the MADRL model.
  • Figure 2: Numerical wave tank configuration and body dimensions.
  • Figure 3: Comparison between SPH simulations and theoretical solutions for wave generation: (a) free-surface elevation for the regular waves and (b) irregular wave spectrum density.
  • Figure 4: Time histories of the simulated heave displacement and velocity of the point absorber together with the experimental data zangHydrodynamicResponsesEfficiency2018.
  • Figure 5: Numerical wave tank setup and point absorber array layout.
  • ...and 12 more figures