Depth-Constrained ASV Navigation with Deep RL and Limited Sensing

TL;DR

The paper tackles depth-constrained autonomous surface vehicle navigation under partial observability, where only a single depth measurement per timestep is available from a downward SBES. It introduces a reinforcement learning framework augmented by localized Gaussian Process regression to progressively estimate a bathymetric map, yielding a belief state that informs decision-making. A covariance-based confidence proxy and gradient-based extrapolation mitigate GP variance saturation, improving learning and sim-to-real transfer. The approach is validated in simulation and on a real ASV, showing improved depth-aware navigation and robustness with minimal sensing hardware. This work advances practical, safe ASV operations in shallow waters by integrating probabilistic environmental estimation directly into the RL policy.

Abstract

Autonomous Surface Vehicles (ASVs) play a crucial role in maritime operations, yet their navigation in shallow-water environments remains challenging due to dynamic disturbances and depth constraints. Traditional navigation strategies struggle with limited sensor information, making safe and efficient operation difficult. In this paper, we propose a reinforcement learning (RL) framework for ASV navigation under depth constraints, where the vehicle must reach a target while avoiding unsafe areas with only a single depth measurement per timestep from a downward-facing Single Beam Echosounder (SBES). To enhance environmental awareness, we integrate Gaussian Process (GP) regression into the RL framework, enabling the agent to progressively estimate a bathymetric depth map from sparse sonar readings. This approach improves decision-making by providing a richer representation of the environment. Furthermore, we demonstrate effective sim-to-real transfer, ensuring that trained policies generalize well to real-world aquatic conditions. Experimental results validate our method's capability to improve ASV navigation performance while maintaining safety in challenging shallow-water environments.

Figures (8)

• Figure 1: Illustration of the improved observation space achieved through GP.
• Figure 2: An image showing the real ASV used for real-world experiments.
• Figure 3: Example of generated depth maps. The red dot marks the target point, while the red dashed line indicates the $L_d$ level set contour, defining the unsafe area. The outermost contour represents the shoreline. The zoomed-in section shows the ASV navigating near shallow waters.
• Figure 4: Illustration of the observation vector and network architecture used.
• Figure 5: Illustration of $r_{\rm depth}$.