Robot Goes Fishing: Rapid, High-Resolution Biological Hotspot Mapping in Coral Reefs with Vision-Guided Autonomous Underwater Vehicles

TL;DR

The paper addresses the need for rapid, fine-grained coral reef monitoring by leveraging a vision-guided autonomous underwater vehicle (AUV) workflow that combines photogrammetry-derived 3D reef topography with frame-level fish detections. Using a downward-facing camera, the AUV collects $4K$ imagery at $6\,\mathrm{FPS}$ from a fixed $2\,\mathrm{m}$ altitude to build $SE(3)$-registered 3D reconstructions and 2D orthomosaics, enabling hotspot maps at sub-meter resolution (approaching $<1\,\mathrm{m}$) and rugosity estimates from the mesh. Fish detections are performed with YOLOv5m and evaluated under in-sample interpolation and out-of-sample transfer using a MegaFishDetector pre-trained on public datasets, revealing strong in-sample performance ($mAP_{50}\approx0.895$) but weaker out-of-sample generalization ($mAP_{50}\approx0.127$ for JS alone, rising to $mAP_{50}\approx0.225$ with JS$_{MFD}$ on TK). The work demonstrates a practical, scalable approach for rapid reef monitoring and hotspot prioritization, while highlighting the need for improved uncertainty quantification and detector robustness for deployment on unseen reefs.

Abstract

Coral reefs are fast-changing and complex ecosystems that are crucial to monitor and study. Biological hotspot detection can help coral reef managers prioritize limited resources for monitoring and intervention tasks. Here, we explore the use of autonomous underwater vehicles (AUVs) with cameras, coupled with visual detectors and photogrammetry, to map and identify these hotspots. This approach can provide high spatial resolution information in fast feedback cycles. To the best of our knowledge, we present one of the first attempts at using an AUV to gather visually-observed, fine-grain biological hotspot maps in concert with topography of a coral reefs. Our hotspot maps correlate with rugosity, an established proxy metric for coral reef biodiversity and abundance, as well as with our visual inspections of the 3D reconstruction. We also investigate issues of scaling this approach when applied to new reefs by using these visual detectors pre-trained on large public datasets.

Figures (2)

• Figure 1: Hotspot detection at two unique coral reef sites in the US Virgin Islands --- Joel's Shoal (JS) and Tektite (TK). From visual surveys collected by an AUV, we generate hotspot maps of fish counts along the robot trajectory and plotted with a log-colorscale (a/d). Our fish detector is fine-tuned on labelled images from JS, but not from TK. We can see distinct hotspots at both reefs, correlated with distinct reef structure visible in our 3D reconstruction, an overhanging structure and a pillar coral (b), as well as with rugosity (c), which is a measure of topographic complexity used by the biological community as a proxy for biodiversity and abundance. Our system enables rapid deployment of an AUV over a predefined but unknown area to identify biologically interesting hotspots.
• Figure 2: Qualitative examples of YOLO output on in-sample and out-of-sample data. In red are predictions with confidence numbers, green are false-negatives (undercounted), magenta are false-positives (overcounted).