Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation

Luis F W Batista, Junghwan Ro, Antoine Richard, Pete Schroepfer, Seth Hutchinson, Cedric Pradalier

TL;DR

The paper tackles the sim-to-real gap in DRL-controlled ASVs by integrating buoyancy and hydrodynamics into a GPU-accelerated RL framework and by employing system identification with domain randomization to improve real-world transfer. Through both simulation and real-world experiments on floating-waste capture, the authors show that SID-DR reduces energy consumption by 13.1% and accelerates task completion by 7.4% compared to nominal approaches, underscoring the practical benefits of more accurate hydrodynamic modeling and robust training. The contributions include an open-source, parallelizable simulation platform and a demonstrated methodology for aligning simulated dynamics with real-world behavior, enhancing efficiency and versatility of autonomous marine operations. These results have potential environmental and operational impact by enabling more capable and energy-efficient ASVs for cleaning and monitoring aquatic environments.

Abstract

Despite the increasing adoption of Deep Reinforcement Learning (DRL) for Autonomous Surface Vehicles (ASVs), there still remain challenges limiting real-world deployment. In this paper, we first integrate buoyancy and hydrodynamics models into a modern Reinforcement Learning framework to reduce training time. Next, we show how system identification coupled with domain randomization improves the RL agent performance and narrows the sim-to-real gap. Real-world experiments for the task of capturing floating waste show that our approach lowers energy consumption by 13.1\% while reducing task completion time by 7.4\%. These findings, supported by sharing our open-source implementation, hold the potential to impact the efficiency and versatility of ASVs, contributing to environmental conservation efforts.

A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation

TL;DR

The paper tackles the sim-to-real gap in DRL-controlled ASVs by integrating buoyancy and hydrodynamics into a GPU-accelerated RL framework and by employing system identification with domain randomization to improve real-world transfer. Through both simulation and real-world experiments on floating-waste capture, the authors show that SID-DR reduces energy consumption by 13.1% and accelerates task completion by 7.4% compared to nominal approaches, underscoring the practical benefits of more accurate hydrodynamic modeling and robust training. The contributions include an open-source, parallelizable simulation platform and a demonstrated methodology for aligning simulated dynamics with real-world behavior, enhancing efficiency and versatility of autonomous marine operations. These results have potential environmental and operational impact by enabling more capable and energy-efficient ASVs for cleaning and monitoring aquatic environments.

Abstract

Despite the increasing adoption of Deep Reinforcement Learning (DRL) for Autonomous Surface Vehicles (ASVs), there still remain challenges limiting real-world deployment. In this paper, we first integrate buoyancy and hydrodynamics models into a modern Reinforcement Learning framework to reduce training time. Next, we show how system identification coupled with domain randomization improves the RL agent performance and narrows the sim-to-real gap. Real-world experiments for the task of capturing floating waste show that our approach lowers energy consumption by 13.1\% while reducing task completion time by 7.4\%. These findings, supported by sharing our open-source implementation, hold the potential to impact the efficiency and versatility of ASVs, contributing to environmental conservation efforts.
Paper Structure (18 sections, 13 equations, 8 figures, 5 tables)

This paper contains 18 sections, 13 equations, 8 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. Capture task
  5. Dynamic Model
  6. Methodology
  7. System Identification
  8. RL Framework for ASV
  9. Training process
  10. Simulation Environment
  11. Autonomous Surface Vehicle Platform Integration
  12. Experiments
  13. Simulation Evaluation
  14. Real-world Evaluation
  15. Evaluation Metrics
  16. ...and 3 more sections

Figures (8)

  • Figure 1: Simulated and Real ASV Platforms Used Through our Experiments
  • Figure 2: Integrated pipeline for DRL-Based ASV Control consisting of four main steps. 1. Real-world experiments for system identification; 2. Train policy in simulation; 3. Tests using an independent simulation environment; 4. Real-world tests.
  • Figure 3: System Identification Experiments. (a) shows surge velocity of the ASV during the acceleration test, and (b) shows the yaw rate of the ASV during the rotation test
  • Figure 4: Comparison of Nominal and Measured Thruster Forces
  • Figure 5: Diagram of the ASV's observation space components
  • ...and 3 more figures