IBURD: Image Blending for Underwater Robotic Detection

TL;DR

IBURD tackles the data scarcity and challenging visuals of underwater debris detection by generating realistic, pixel-annotated synthetic images through a two-pass image blending pipeline that combines Poisson editing with style transfer. An adaptive, FFT-based blurriness measure tunes the style loss to preserve object content while matching underwater appearance, and source objects are produced semi-automatically via DALL‑E and SAM with COCO-format annotations. Evaluations on the TrashCan dataset and real LoCO AUV deployments show IBURD-augmented data improves detection and instance segmentation performance and enables on-board autonomous operation in data-scarce underwater environments. The work demonstrates a practical path to scalable underwater debris detection via synthetic data, with potential impact on environmental cleanup missions using AUVs.

Abstract

We present an image blending pipeline, \textit{IBURD}, that creates realistic synthetic images to assist in the training of deep detectors for use on underwater autonomous vehicles (AUVs) for marine debris detection tasks. Specifically, IBURD generates both images of underwater debris and their pixel-level annotations, using source images of debris objects, their annotations, and target background images of marine environments. With Poisson editing and style transfer techniques, IBURD is even able to robustly blend transparent objects into arbitrary backgrounds and automatically adjust the style of blended images using the blurriness metric of target background images. These generated images of marine debris in actual underwater backgrounds address the data scarcity and data variety problems faced by deep-learned vision algorithms in challenging underwater conditions, and can enable the use of AUVs for environmental cleanup missions. Both quantitative and robotic evaluations of IBURD demonstrate the efficacy of the proposed approach for robotic detection of marine debris.

Figures (7)

• Figure 1: Demonstration of object detection on board the LoCO AUV loco_paper_2020 in the Caribbean sea. The detection model is only trained on a synthetic dataset generated with our proposed pipeline. Magenta denotes starfish detection and green denotes can detection, where the color corresponds to the illuminated LEDs on the left "eye" fulton2023hreyes of the robot. Both objects are successfully detected.
• Figure 2: Comparison of generated images using three approaches: Poisson image editing perez_poisson_2003, Deep image blending zhang_deep_2020 and our method, IBURD. In our approach, we can successfully prevent over-stylization of the blended objects.
• Figure 3: IBURD pipeline: The source image with its pixel-level annotation and background image are fed as inputs. The first pass resizes and rotates the source object and selects a location for blending it. Poisson editing smooths the boundary between the object and the background. The second pass changes the style to produce the final image. In the zoomed-in region of the object, the style difference between the appearance of the first and second pass image is visible.
• Figure 4: Object content is lost when blending objects in a blurry background. (a) shows a gradual content loss on increasing style loss weight. (b) shows examples of blurry backgrounds.
• Figure 5: Sample images generated from IBURD. The first column shows images blended on pool backgrounds. The second column contains objects blended on ocean backgrounds.