Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reinforcement-learning robotic sailboats: simulator and preliminary results

Eduardo Charles Vasconcellos, Ronald M Sampaio, André P D Araújo, Esteban Walter Gonzales Clua, Philippe Preux, Raphael Guerra, Luiz M G Gonçalves, Luis Martí, Hernan Lira, Nayat Sanchez-Pi

TL;DR

This paper addresses the challenge of creating a realistic virtual oceanic environment to develop and test reinforcement learning for Unmanned Surface Vehicles (USVs), focusing on sailing robots. It outlines core simulator features—accurate physics, ROS integration, realistic sensors/actuators, and seamless digital twin integration—and details a physics framework that includes hydrodynamic, hydrostatic, and aerodynamic forces within a 6-DOF motion model, using hull-part buoyancy segmentation. An E-Boat digital twin is built with a CAD model, mass properties, and wind/water force models, enabling RL experiments where a PPO policy navigates a six-waypoint mission in Gazebo (with Omniverse explored for rendering). The results demonstrate consistent maneuvers across multiple runs and highlight the platform as a practical tool for advancing autonomous sailing robotics, while noting tradeoffs between Gazebo’s robotics focus and Omniverse’s rendering capabilities for future work and algorithm comparison.

Abstract

This work focuses on the main challenges and problems in developing a virtual oceanic environment reproducing real experiments using Unmanned Surface Vehicles (USV) digital twins. We introduce the key features for building virtual worlds, considering using Reinforcement Learning (RL) agents for autonomous navigation and control. With this in mind, the main problems concern the definition of the simulation equations (physics and mathematics), their effective implementation, and how to include strategies for simulated control and perception (sensors) to be used with RL. We present the modeling, implementation steps, and challenges required to create a functional digital twin based on a real robotic sailing vessel. The application is immediate for developing navigation algorithms based on RL to be applied on real boats.

Reinforcement-learning robotic sailboats: simulator and preliminary results

TL;DR

This paper addresses the challenge of creating a realistic virtual oceanic environment to develop and test reinforcement learning for Unmanned Surface Vehicles (USVs), focusing on sailing robots. It outlines core simulator features—accurate physics, ROS integration, realistic sensors/actuators, and seamless digital twin integration—and details a physics framework that includes hydrodynamic, hydrostatic, and aerodynamic forces within a 6-DOF motion model, using hull-part buoyancy segmentation. An E-Boat digital twin is built with a CAD model, mass properties, and wind/water force models, enabling RL experiments where a PPO policy navigates a six-waypoint mission in Gazebo (with Omniverse explored for rendering). The results demonstrate consistent maneuvers across multiple runs and highlight the platform as a practical tool for advancing autonomous sailing robotics, while noting tradeoffs between Gazebo’s robotics focus and Omniverse’s rendering capabilities for future work and algorithm comparison.

Abstract

This work focuses on the main challenges and problems in developing a virtual oceanic environment reproducing real experiments using Unmanned Surface Vehicles (USV) digital twins. We introduce the key features for building virtual worlds, considering using Reinforcement Learning (RL) agents for autonomous navigation and control. With this in mind, the main problems concern the definition of the simulation equations (physics and mathematics), their effective implementation, and how to include strategies for simulated control and perception (sensors) to be used with RL. We present the modeling, implementation steps, and challenges required to create a functional digital twin based on a real robotic sailing vessel. The application is immediate for developing navigation algorithms based on RL to be applied on real boats.
Paper Structure (8 sections, 1 equation, 3 figures)

This paper contains 8 sections, 1 equation, 3 figures.

Table of Contents

  1. Introduction
  2. Key features on building a reliable simulator for sailing robots
  3. Physics
  4. ROS support
  5. Sensors and actuators
  6. Integration of USV digital twin into the environment
  7. Towards autonomous navigation and control with reinforcement learning
  8. Conclusions

Figures (3)

  • Figure 1: E-Boat physics characterization.
  • Figure 2: Sea surface rendering in Omniverse and Gazebo.
  • Figure 3: RL agent policy behavior in twenty runs of the same mission. The mission path is defined as BCDACA.