MarineGym: Accelerated Training for Underwater Vehicles with High-Fidelity RL Simulation

TL;DR

MarineGym is introduced, a novel simulation framework designed to enhance RL training efficiency for UUVs by utilizing GPU acceleration, and sets the stage for advancing RL in underwater robotics and facilitating efficient training in complex, dynamic environments.

Abstract

Reinforcement Learning (RL) is a promising solution, allowing Unmanned Underwater Vehicles (UUVs) to learn optimal behaviors through trial and error. However, existing simulators lack efficient integration with RL methods, limiting training scalability and performance. This paper introduces MarineGym, a novel simulation framework designed to enhance RL training efficiency for UUVs by utilizing GPU acceleration. MarineGym offers a 10,000-fold performance improvement over real-time simulation on a single GPU, enabling rapid training of RL algorithms across multiple underwater tasks. Key features include realistic dynamic modeling of UUVs, parallel environment execution, and compatibility with popular RL frameworks like PyTorch and TorchRL. The framework is validated through four distinct tasks: station-keeping, circle tracking, helical tracking, and lemniscate tracking. This framework sets the stage for advancing RL in underwater robotics and facilitating efficient training in complex, dynamic environments.

Figures (1)

• Figure 1: The results of training four different tasks in MarineGym. (a) Rendered simulation scene during the validation process of the trained agent. (b) Return over training time. (c) Position error over training time.