Cross-view and Cross-domain Underwater Localization based on Optical Aerial and Acoustic Underwater Images

TL;DR

The method identifies the correlation between color aerial images and underwater acoustic images to improve the localization of underwater vehicles that travel in partially structured environments such as harbors and marinas.

Abstract

Cross-view image matches have been widely explored on terrestrial image localization using aerial images from drones or satellites. This study expands the cross-view image match idea and proposes a cross-domain and cross-view localization framework. The method identifies the correlation between color aerial images and underwater acoustic images to improve the localization of underwater vehicles that travel in partially structured environments such as harbors and marinas. The approach is validated on a real dataset acquired by an underwater vehicle in a marina. The results show an improvement in the localization when compared to the dead reckoning of the vehicle.

Figures (6)

• Figure 1: An underwater vehicle doted on multi-beam forward-looking sonar travels in a semi-structured marina environment. The localization is estimated using structures such as the pier and the shoreline, both visible on the range of the acoustic images. However, the vehicle gets lost when it moves to an open area where no structures are available. Aerial images of the environment can improve the localization and re-localize the vehicle when it returns to the shoreline region. Features highlighted in green and red are adopted in a Cross-view and Cross-domain matching system. A particle filter framework fuses both images and estimates the vehicle location using a Deep Neural Network (DNN) as an observation model. The system belief is built on aerial images.
• Figure 2: Proposed framework for cross-view (top and frontal) cross-domain (optical and acoustic) vehicle localization using acoustic and satellite images. Initially, the satellite image is semantically segmented in an offline process. The Underwater Acoustic Image Processing performs image enhancement based on the alignment of a batch of acoustic images $O_t = \{ o_{t},o_{t-1},...,o_{t-n} \}$. The particle filter adopts the satellite image as the map $\mathbb{M}$, the enhanced acoustic images $\hat{o}_{t}$ as the observations and the vehicle odometry as control signal $u_t$ and estimates the underwater vehicle state $Y_t$.
• Figure 3: A particle state $s_{t}^{k}$ is converted into a crop of the semantically segmented satellite image. The crop has the same shape and scale as the acoustic images.
• Figure 4: On the dataset ARACATI 2017, a floating board holds a DGPS on surface water and a Seabotix LBV 300 underwater ROV. It allows collecting precise position data as well as underwater acoustic images.
• Figure 5: Vehicle path in meters estimated by dead reckoning in red, particle filter in blue, and DGPS reference in green. The colored circles highlight timestamps when features were not available on sonar or when the method starts to correct the localization based on cross-view and cross-domain matching.