Sonar-based Deep Learning in Underwater Robotics: Overview, Robustness and Challenges

TL;DR

This paper addresses the safety-critical problem of deploying real-time sonar-based deep learning on AUVs by providing a robustness-focused survey of the field. It synthesizes open-source datasets, underwater simulators, and robustness techniques (including neural network verification, adversarial defenses, and OOD/uncertainty quantification) to map the current landscape and gaps. A key contribution is the proposed OpenSonarDatasets repository and a structured robustness workflow to bridge training and deployment under underwater uncertainty. The work highlights the sim-to-real gap, emphasizes the need for standardized baselines and evaluation under diverse sonar conditions, and outlines practical paths toward safer, more reliable onboard sonar perception for underwater missions.

Abstract

With the growing interest in underwater exploration and monitoring, Autonomous Underwater Vehicles (AUVs) have become essential. The recent interest in onboard Deep Learning (DL) has advanced real-time environmental interaction capabilities relying on efficient and accurate vision-based DL models. However, the predominant use of sonar in underwater environments, characterized by limited training data and inherent noise, poses challenges to model robustness. This autonomy improvement raises safety concerns for deploying such models during underwater operations, potentially leading to hazardous situations. This paper aims to provide the first comprehensive overview of sonar-based DL under the scope of robustness. It studies sonar-based DL perception task models, such as classification, object detection, segmentation, and SLAM. Furthermore, the paper systematizes sonar-based state-of-the-art datasets, simulators, and robustness methods such as neural network verification, out-of-distribution, and adversarial attacks. This paper highlights the lack of robustness in sonar-based DL research and suggests future research pathways, notably establishing a baseline sonar-based dataset and bridging the simulation-to-reality gap.

Figures (8)

• Figure 1: Sonar perception with Side Scan Sonar (SSS) aubard_2024_10528135 and Forward Looking Sonar (FLS) data_aug_dataset. This figure represents the setup and visual information from the Multi-Beam Echo Sonar (MBES), SSS, and FLS. It shows that while the SSS provides information on past data from the port to the starboard, the FLS gives current information from the front of the AUV and the MBES from beneath the vehicle.
• Figure 2: Samples object detection on sonar images. Those two samples show an object detection model prediction for a vessel on an FLS image and a shipwreck on an SSS image.
• Figure 3: Samples segmentation on sonar images. Those two samples show a segmentation model prediction on FLS and SSS images.
• Figure 4: Samples SLAM on sonar images. Those two samples show a SLAM detected keypoint correspondence prediction on Mechanical Scanning Imaging Sonar (MSIS) and SSS images. Those points refer to identifying specific points of interest (key points) across multiple observations or images considered at the same physical location in the environment.
• Figure 5: Samples of Underwater Sonar Datasets. Those samples illustrate some of the SOA sonar datasets, which represent GeoTiff Adriatic_Reefs, SSS sethuraman2024machine, FLS UXO images with different objects such as pipelines alvareztunon2024subpipe, walls aubard_2024_10528135 and shipwrecks sethuraman2024machine.