UEOF: A Benchmark Dataset for Underwater Event-Based Optical Flow

TL;DR

UEOF addresses the lack of underwater event-based optical-flow data by generating a physically-grounded synthetic benchmark: PBRT RGBD sequences are converted to temporally dense pseudo-events via video-to-event processing, providing dense ground-truth flow, depth, and camera motion. The dataset combines VAROS (static infrastructure with precise pose/IMU) and LOFUE (dynamic scenes with moving objects) to cover diverse underwater dynamics, at two resolutions and thousands of frames, totaling billions of events. Benchmarking both learning-based and model-based optical-flow methods reveals significant domain gaps when transferring terrestrial-trained approaches to underwater imagery, with caustics, turbidity, and low texture degrading performance and highlighting the need for multimodal and physically-informed perception pipelines. Overall, UEOF provides a essential baseline and a pathway for developing robust underwater event-based perception for autonomous underwater vehicles.

Abstract

Underwater imaging is fundamentally challenging due to wavelength-dependent light attenuation, strong scattering from suspended particles, turbidity-induced blur, and non-uniform illumination. These effects impair standard cameras and make ground-truth motion nearly impossible to obtain. On the other hand, event cameras offer microsecond resolution and high dynamic range. Nonetheless, progress on investigating event cameras for underwater environments has been limited due to the lack of datasets that pair realistic underwater optics with accurate optical flow. To address this problem, we introduce the first synthetic underwater benchmark dataset for event-based optical flow derived from physically-based ray-traced RGBD sequences. Using a modern video-to-event pipeline applied to rendered underwater videos, we produce realistic event data streams with dense ground-truth flow, depth, and camera motion. Moreover, we benchmark state-of-the-art learning-based and model-based optical flow prediction methods to understand how underwater light transport affects event formation and motion estimation accuracy. Our dataset establishes a new baseline for future development and evaluation of underwater event-based perception algorithms. The source code and dataset for this project are publicly available at https://robotic-vision-lab.github.io/ueof.

Figures (5)

• Figure 1: An illustration of data and ground-truth modalities from the UEOF dataset. Our dataset assembles physically-realistic underwater RGB imagery, ground-truth optical flow, camera ego-velocities, and temporally dense event streams, enabling the benchmarking of multimodal event-based optical flow estimation algorithms.
• Figure 2: An illustration of key underwater visual degradation modes and their impact on event formation. RGB frames (top row) and the corresponding events (bottom row) are shown for a variety of challenging underwater scenarios.
• Figure 3: The UEOF data generation pipeline. First, Blender is utilized to render high-fidelity RGB frames and extract ground-truth data from the LOFUE and VAROS datasets. Next, the RGB frames undergo event conversion via the v2e toolbox. Finally, the resulting event streams are processed by event-based optical flow methods and the predicted flow is evaluated against the ground-truth data using standard endpoint error metrics.
• Figure 4: Optical flow statistics for the UEOF dataset: (a) the joint distribution of $(u,v)$ pixel displacements highlight dense coverage of motion directions; (b) the semi-log histogram shows a heavy-tailed distribution with a mean of 6.1 px.
• Figure 5: Qualitative event-based optical flow estimation results using the UEOF dataset. Columns (a-e) show the frames, events, ground-truth (GT) optical flow, and the predicted event-based optical flow using a representative MB (MultiCM shiba2022secrets) and LB (MotionPriorCMax hamann2024motion) method. The top five and bottom five rows correspond to the shallow-water and deep-water environment sequences, respectively. The optical flow direction and magnitude are encoded according to the color legend shown below.