Machine Learning for Shipwreck Segmentation from Side Scan Sonar Imagery: Dataset and Benchmark

TL;DR

Underwater shipwreck segmentation suffers from a scarcity of labeled benchmark data. The authors introduce AI4Shipwrecks, a real-world, pixel-wise labeled side scan sonar dataset with 286 images across 28 shipwreck sites collected by an AUV in Thunder Bay National Marine Sanctuary. They provide open-source preprocessing, ground-truth labeling guidelines, and a benchmark of multiple state-of-the-art segmentation models, highlighting practical challenges in sonar data and the feasibility of existing architectures. The work enables reproducible evaluation and points to future directions like synthetic data augmentation and few-shot learning to improve performance with limited underwater data.

Abstract

Open-source benchmark datasets have been a critical component for advancing machine learning for robot perception in terrestrial applications. Benchmark datasets enable the widespread development of state-of-the-art machine learning methods, which require large datasets for training, validation, and thorough comparison to competing approaches. Underwater environments impose several operational challenges that hinder efforts to collect large benchmark datasets for marine robot perception. Furthermore, a low abundance of targets of interest relative to the size of the search space leads to increased time and cost required to collect useful datasets for a specific task. As a result, there is limited availability of labeled benchmark datasets for underwater applications. We present the AI4Shipwrecks dataset, which consists of 28 distinct shipwrecks totaling 286 high-resolution labeled side scan sonar images to advance the state-of-the-art in autonomous sonar image understanding. We leverage the unique abundance of targets in Thunder Bay National Marine Sanctuary in Lake Huron, MI, to collect and compile a sonar imagery benchmark dataset through surveys with an autonomous underwater vehicle (AUV). We consulted with expert marine archaeologists for the labeling of robotically gathered data. We then leverage this dataset to perform benchmark experiments for comparison of state-of-the-art supervised segmentation methods, and we present insights on opportunities and open challenges for the field. The dataset and benchmarking tools will be released as an open-source benchmark dataset to spur innovation in machine learning for Great Lakes and ocean exploration. The dataset and accompanying software are available at https://umfieldrobotics.github.io/ai4shipwrecks/.

Figures (10)

• Figure 1: Our AI4Shipwrecks dataset aims to accelerate the development of shipwreck segmentation algorithms for sonar data collected onboard autonomous systems.
• Figure 2: Data acquisition, processing, and network inference pipeline using the Iver3 autonomous underwater vehicle. The yellow lines in Mission Planning denote the AUV's trajectory, with blue segments occuring at survey depth.
• Figure 3: Map of survey sites in TBNMS, Lake Huron, MI. Callouts include example sonar data overlaid with ground truth labels. Color indicates sites that are included in testing (red) and training (yellow) splits, and locations of additional terrain surveys (green). Best viewed in color and zoomed in.
• Figure 4: Iver3 data collection platform equipped with advanced localization and high-resolution seafloor mapping capabilities.
• Figure 5: a) SSS sensor model detailing sensor tilt angle ($\theta_t$), first bottom return ($p_{fbr}$), nadir gap, and ensonified point on object ($p(P_x, P_y, P_z)$). The SSS field of view is shown in blue. b) The AUV (in yellow) performs a survey with half-swath width ($\frac{s}{2}$), leg width ($\delta_w$), leg length ($l$), and total survey width ($w_s$). For each survey leg, the AUV dives to the depth from surface ($d_s$) or height over ground ($h_g$) and resurfaces between legs to acquire a GPS update. The SSS is only collecting data once submerged at depth ($d_s$) for length ($l$).