AquaFuse: Waterbody Fusion for Physics Guided View Synthesis of Underwater Scenes

TL;DR

Underwater imaging suffers from scattering, absorption, and backscatter, complicating data augmentation and view synthesis. AquaFuse introduces a physics-guided waterbody fusion approach that transfers waterbody appearance between scenes while preserving depth and object geometry, enabled by a closed-form fusion based on the revised underwater image formation model. The method combines real-time depth estimation, backscatter/veiling-light parameter estimation, and depth-aware fusion to achieve depth-consistency above 94% and SSIM around 0.90–0.95 across depths, while enabling accurate 3D view synthesis via Gaussian Splatting. With CPU inference reaching up to 342 FPS for 256×256 (and 22 FPS on a Raspberry Pi), AquaFuse supports real-time, geometry-preserving data augmentation and enhancement for underwater perception and robotics.

Abstract

We introduce the idea of AquaFuse, a physics-based method for synthesizing waterbody properties in underwater imagery. We formulate a closed-form solution for waterbody fusion that facilitates realistic data augmentation and geometrically consistent underwater scene rendering. AquaFuse leverages the physical characteristics of light propagation underwater to synthesize the waterbody from one scene to the object contents of another. Unlike data-driven style transfer, AquaFuse preserves the depth consistency and object geometry in an input scene. We validate this unique feature by comprehensive experiments over diverse underwater scenes. We find that the AquaFused images preserve over 94% depth consistency and 90-95% structural similarity of the input scenes. We also demonstrate that it generates accurate 3D view synthesis by preserving object geometry while adapting to the inherent waterbody fusion process. AquaFuse opens up a new research direction in data augmentation by geometry-preserving style transfer for underwater imaging and robot vision applications.

Figures (10)

• Figure 1: AquaFuse is a physics-guided waterbody fusion method to fuse waterbody properties of a reference image to an input image. It leverages closed-form solutions to estimate and exploit the scene depth, background light, backscatter, and medium illumination parameters for guiding the fusion process. As shown, AquaFused scenes are perceptually realistic and also meet the underwater image formation constraints.
• Figure 2: An illustrative example of our image restoration pipeline; (a) input signal $\mathbf U$; (b) ground truth depth; (c) estimated depth $\mathbf Z$; (d,e) estimated illumination map for $R$ and $M$ channels; and (f) the restored signal $\mathbf I$.
• Figure 3: Computational outline of AquaFuse with examples.
• Figure 4: Our evaluation samples include comprehensive data collected on six underwater sites with incremental depth intervals ($10'$-$80'$). We also use reference images from a few existing benchmark datasets.
• Figure 5: Two examples of waterbody crossover by AquaFuse are shown for physics-aware data augmentation; note that this process enables $2\times$ data generation.