Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EREBUS: End-to-end Robust Event Based Underwater Simulation

Hitesh Kyatham, Arjun Suresh, Aadi Palnitkar, Yiannis Aloimonos

TL;DR

Underwater perception is hindered by poor lighting and turbidity; this work introduces EREBUS, a full end-to-end pipeline that generates realistic synthetic event data from Blender-rendered underwater scenes and uses a DVS simulator to produce event streams. It demonstrates rock detection under challenging visibility using few-shot learning with YOLOv8-n on DVS frames, achieving a mAP of 0.83 with only 10 training images across 250 epochs. Key contributions include a Blender-based underwater event-data generator, a few-shot training workflow for event data, particle-noise robustness augmentation, and an extensible framework aimed at public datasets and benchmarks. The approach enables rapid bootstrapping of robust, event-based perception for AUVs in challenging marine environments, reducing reliance on scarce real underwater event data.

Abstract

The underwater domain presents a vast array of challenges for roboticists and computer vision researchers alike, such as poor lighting conditions and high dynamic range scenes. In these adverse conditions, traditional vision techniques struggle to adapt and lead to suboptimal performance. Event-based cameras present an attractive solution to this problem, mitigating the issues of traditional cameras by tracking changes in the footage on a frame-by-frame basis. In this paper, we introduce a pipeline which can be used to generate realistic synthetic data of an event-based camera mounted to an AUV (Autonomous Underwater Vehicle) in an underwater environment for training vision models. We demonstrate the effectiveness of our pipeline using the task of rock detection with poor visibility and suspended particulate matter, but the approach can be generalized to other underwater tasks.

EREBUS: End-to-end Robust Event Based Underwater Simulation

TL;DR

Underwater perception is hindered by poor lighting and turbidity; this work introduces EREBUS, a full end-to-end pipeline that generates realistic synthetic event data from Blender-rendered underwater scenes and uses a DVS simulator to produce event streams. It demonstrates rock detection under challenging visibility using few-shot learning with YOLOv8-n on DVS frames, achieving a mAP of 0.83 with only 10 training images across 250 epochs. Key contributions include a Blender-based underwater event-data generator, a few-shot training workflow for event data, particle-noise robustness augmentation, and an extensible framework aimed at public datasets and benchmarks. The approach enables rapid bootstrapping of robust, event-based perception for AUVs in challenging marine environments, reducing reliance on scarce real underwater event data.

Abstract

The underwater domain presents a vast array of challenges for roboticists and computer vision researchers alike, such as poor lighting conditions and high dynamic range scenes. In these adverse conditions, traditional vision techniques struggle to adapt and lead to suboptimal performance. Event-based cameras present an attractive solution to this problem, mitigating the issues of traditional cameras by tracking changes in the footage on a frame-by-frame basis. In this paper, we introduce a pipeline which can be used to generate realistic synthetic data of an event-based camera mounted to an AUV (Autonomous Underwater Vehicle) in an underwater environment for training vision models. We demonstrate the effectiveness of our pipeline using the task of rock detection with poor visibility and suspended particulate matter, but the approach can be generalized to other underwater tasks.

Paper Structure

This paper contains 14 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Underwater Simulations
  4. Challenges of the Underwater Domain
  5. Event Based Cameras
  6. Applications of Event Cameras to Underwater Domain
  7. Event Based Simulations
  8. Methodology
  9. Underwater Scene Simulation in Blender
  10. Event Stream Generation
  11. Object Segmentation using YOLO with Few-Shot Learning
  12. Enhancing Robustness through Simulated Particle Noise and Adaptive Training
  13. Proposed Pipeline Summary
  14. Conclusions and Future Work

Figures (3)

  • Figure 1: Illustration of EREBUS frame-by-frame image processing
  • Figure 2: A depiction of the pipeline of EREBUS. The Blender generated RGB image was brightened for demonstration purposes.
  • Figure 3: Illustration showing particle sizes ranging from the default to 8× larger. Best viewed at 400% zoom. Note: the background is not completely black.