Deep Learning-Enhanced Visual Monitoring in Hazardous Underwater Environments with a Swarm of Micro-Robots

TL;DR

This work tackles automated long-term underwater monitoring in hazardous settings using a swarm of micro-robots (Bubble) to build a coherent spatio-temporal view despite fluid-induced drift and rotation. It introduces a synthetic dataset (SFP10) and a multi-modal deep-learning pipeline that combines snapshot imagery, global context masks, and noisy coordinate data to predict corrected coordinates and reassemble images. The approach demonstrates high coordinate accuracy and plausible image stitching on synthetic data, significantly improving alignment under challenging motion. By releasing both the dataset and code, the authors enable broader research toward safer, more efficient monitoring in extreme environments and potential real-time deployment.

Abstract

Long-term monitoring and exploration of extreme environments, such as underwater storage facilities, is costly, labor-intensive, and hazardous. Automating this process with low-cost, collaborative robots can greatly improve efficiency. These robots capture images from different positions, which must be processed simultaneously to create a spatio-temporal model of the facility. In this paper, we propose a novel approach that integrates data simulation, a multi-modal deep learning network for coordinate prediction, and image reassembly to address the challenges posed by environmental disturbances causing drift and rotation in the robots' positions and orientations. Our approach enhances the precision of alignment in noisy environments by integrating visual information from snapshots, global positional context from masks, and noisy coordinates. We validate our method through extensive experiments using synthetic data that simulate real-world robotic operations in underwater settings. The results demonstrate very high coordinate prediction accuracy and plausible image assembly, indicating the real-world applicability of our approach. The assembled images provide clear and coherent views of the underwater environment for effective monitoring and inspection, showcasing the potential for broader use in extreme settings, further contributing to improved safety, efficiency, and cost reduction in hazardous field monitoring. Code is available on https://github.com/ChrisChen1023/Micro-Robot-Swarm.

Figures (5)

• Figure 1: The overarching vision of the proposed visual exploration system involves deploying a swarm of micro-surface robots, Bubbles.
• Figure 2: (a) The electronics of Bubble, divided into two PCBs with different functionalities. (b) The system architecture of the robot illustrates its key functions. The section within the dashed rectangle represents the top PCB.
• Figure 3: Image simulating the spent fuel pond; hollow circles indicate empty rods and solid circles indicate full rods.
• Figure 4: Overview of the overall pipeline, which integrates data simulation (Left), noisy coordinate correction via a deep learning network (Middle), and image reassembly (Right) to generate coherent images from snapshots captured by micro-robots in noisy underwater environments.
• Figure 5: The first two rows correspond to the first example, and the bottom two rows correspond to the second example. The visual comparisons demonstrate that our results exhibit a more coherent and consistent structure.