Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Robotics

Shuguang Chu, Zebin Huang, Yutong Li, Mingwei Lin, Ignacio Carlucho, Yvan R. Petillot, Canjun Yang

TL;DR

MarineGym addresses the challenges of applying reinforcement learning to underwater robotics by delivering a GPU-accelerated, RL-friendly simulation platform with a diverse UUV model library and a configurable domain-randomization toolkit. It includes a standardized benchmarking suite to enable fair comparisons and Sim2Sim transfer to real-world deployments. The experiments demonstrate high simulation throughput (up to 2.5e5 FPS on a single RTX 3060) and improved policy robustness through domain randomization, including substantial improvements in docking under novel conditions. While promising for scalable RL research in underwater robotics, the work notes the need for real-world validation and richer underwater visual and task realism in future work.

Abstract

This work presents the MarineGym, a high-performance reinforcement learning (RL) platform specifically designed for underwater robotics. It aims to address the limitations of existing underwater simulation environments in terms of RL compatibility, training efficiency, and standardized benchmarking. MarineGym integrates a proposed GPU-accelerated hydrodynamic plugin based on Isaac Sim, achieving a rollout speed of 250,000 frames per second on a single NVIDIA RTX 3060 GPU. It also provides five models of unmanned underwater vehicles (UUVs), multiple propulsion systems, and a set of predefined tasks covering core underwater control challenges. Additionally, the DR toolkit allows flexible adjustments of simulation and task parameters during training to improve Sim2Real transfer. Further benchmark experiments demonstrate that MarineGym improves training efficiency over existing platforms and supports robust policy adaptation under various perturbations. We expect this platform could drive further advancements in RL research for underwater robotics. For more details about MarineGym and its applications, please visit our project page: https://marine-gym.com/.

MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Robotics

TL;DR

MarineGym addresses the challenges of applying reinforcement learning to underwater robotics by delivering a GPU-accelerated, RL-friendly simulation platform with a diverse UUV model library and a configurable domain-randomization toolkit. It includes a standardized benchmarking suite to enable fair comparisons and Sim2Sim transfer to real-world deployments. The experiments demonstrate high simulation throughput (up to 2.5e5 FPS on a single RTX 3060) and improved policy robustness through domain randomization, including substantial improvements in docking under novel conditions. While promising for scalable RL research in underwater robotics, the work notes the need for real-world validation and richer underwater visual and task realism in future work.

Abstract

This work presents the MarineGym, a high-performance reinforcement learning (RL) platform specifically designed for underwater robotics. It aims to address the limitations of existing underwater simulation environments in terms of RL compatibility, training efficiency, and standardized benchmarking. MarineGym integrates a proposed GPU-accelerated hydrodynamic plugin based on Isaac Sim, achieving a rollout speed of 250,000 frames per second on a single NVIDIA RTX 3060 GPU. It also provides five models of unmanned underwater vehicles (UUVs), multiple propulsion systems, and a set of predefined tasks covering core underwater control challenges. Additionally, the DR toolkit allows flexible adjustments of simulation and task parameters during training to improve Sim2Real transfer. Further benchmark experiments demonstrate that MarineGym improves training efficiency over existing platforms and supports robust policy adaptation under various perturbations. We expect this platform could drive further advancements in RL research for underwater robotics. For more details about MarineGym and its applications, please visit our project page: https://marine-gym.com/.

Paper Structure

This paper contains 17 sections, 3 equations, 3 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Simulated Environments for Underwater Robot Learning
  4. Reinforcement Learning for Underwater Robots
  5. Station-Keeping
  6. Tracking
  7. Docking
  8. METHOD
  9. System architecture
  10. GPU-accelerated underwater simulation pipeline
  11. DR toolkit
  12. Multi-type UUV model library
  13. Experiments
  14. Simulation speed comparative
  15. Benchmarking on UUV tasks
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Overview of the MarineGym platform featuring underwater robot learning environments. Clockwise from the top left: large-scale multi-robot scenario, underwater robot models, a docking task, and a station-keeping task.
  • Figure 2: The architecture of MarineGym.
  • Figure 3: Learning curves of five UUV models across three tasks under Standard (blue), Disturbance (orange), and Disturbance + Randomization (green) environments, each trained with four different random seeds. Each row represents a different task (Station keeping, Trajectory tracking, and Docking), while each column corresponds to a specific UUV model (BlueROV, BlueROV Heavy, HAUV, LAUV, iAUV).