Aquatic-GS: A Hybrid 3D Representation for Underwater Scenes

TL;DR

Aquatic-GS tackles the challenge of simultaneous water and object representation in underwater scenes by pairing a Neural Water Field (NWF) that models spatially varying water parameters with explicit 3D Gaussian Splatting (3DGS) for the scene geometry. The method integrates these components through a physics-based underwater image formation model $I = J e^{-oldsymbol{\beta}^D \boldsymbol{R}} + \mathbf{A}(1 - e^{-oldsymbol{\beta}^B \boldsymbol{R}})$ and introduces a Depth-Guided Optimization framework to refine geometry, especially at long distances. Key contributions include non-uniform learning of water parameter distributions, four depth-guided loss terms (transmittance regularization, depth variance minimization, inverse depth correlation, and patch frequency), and a practical training-and-rendering pipeline that achieves 410× faster real-time rendering while improving underwater novel-view synthesis and image restoration. The results demonstrate superior rendering quality and restoration stability across real and simulated datasets, with code and data publicly available, highlighting the practical impact for underwater robotics, marine science, and immersive underwater visualization.

Abstract

Representing underwater 3D scenes is a valuable yet complex task, as attenuation and scattering effects during underwater imaging significantly couple the information of the objects and the water. This coupling presents a significant challenge for existing methods in effectively representing both the objects and the water medium simultaneously. To address this challenge, we propose Aquatic-GS, a hybrid 3D representation approach for underwater scenes that effectively represents both the objects and the water medium. Specifically, we construct a Neural Water Field (NWF) to implicitly model the water parameters, while extending the latest 3D Gaussian Splatting (3DGS) to model the objects explicitly. Both components are integrated through a physics-based underwater image formation model to represent complex underwater scenes. Moreover, to construct more precise scene geometry and details, we design a Depth-Guided Optimization (DGO) mechanism that uses a pseudo-depth map as auxiliary guidance. After optimization, Aquatic-GS enables the rendering of novel underwater viewpoints and supports restoring the true appearance of underwater scenes, as if the water medium were absent. Extensive experiments on both simulated and real-world datasets demonstrate that Aquatic-GS surpasses state-of-the-art underwater 3D representation methods, achieving better rendering quality and real-time rendering performance with a 410x increase in speed. Furthermore, regarding underwater image restoration, Aquatic-GS outperforms representative dewatering methods in color correction, detail recovery, and stability. Our models, code, and datasets can be accessed at https://aquaticgs.github.io.

Figures (7)

• Figure 1: (a) Underwater imaging environment and the hybrid representation strategy employed by Aquatic-GS. (b) Scene information learned by Aquatic-GS, including water parameters, the true appearance, and the geometry of the underwater scene. 'Atten.' and 'Coeffs' are abbreviations for Attenuation and Coefficients, respectively. (c) Rendering of an underwater image by Aquatic-GS using a physics-based underwater imaging model.
• Figure 2: Pipeline of Aquatic-GS. For a given viewpoint within a bounded viewing frustum, Aquatic-GS uses the Neural Water Field to obtain the underwater ambient light $\bm{A}$, attenuation coefficients $\bm{\beta}^D$, and backscattering coefficients $\bm{\beta}^B$ for the current viewpoint, while utilizing 3D Gaussian Splatting to render the water-free image $\bm{J}$ along with the corresponding depth $\bm{D}$ and distance maps $\bm{R}$. These outputs are then unified through a physics-based underwater image formation (UIF) model to render the corresponding underwater image $\bm{I}$. During the optimization, along with the reconstruction loss between $\bm{I}$ and $\hat{\bm{I}}$, the Depth-Guided Optimization mechanism, which includes four specifically designed loss functions, leverages a pseudo-depth map $\hat{\bm{D}}$ to guide Aquatic-GS in producing more precise scene representation.
• Figure 3: Visual comparison of the water-free scene learned with reconstruction loss alone versus those learned using our DGO mechanism. The former displays a haze, especially in distant areas, as highlighted by the red and green boxes. The introduction of our DGO mechanism effectively reduces haze and restores more geometry details in these challenging regions.
• Figure 4: Qualitative comparison in the underwater NVS task. Alongside the rendered underwater images, corresponding rendered depth maps are also displayed to assess the geometric representation capabilities of different approaches. Challenging regions are highlighted within red boxes. The pseudo-depth maps serve as benchmarks for evaluating geometric fidelity. Aquatic-GS demonstrates superior rendering quality, effectively reducing blur and artifacts, while reconstructing more details, particularly in distant regions. Regarding geometric representation, our Aquatic-GS generates more reasonable depth maps that are smooth with clear edges and minimal influence from floaters.
• Figure 5: Hue values vs. distance. We tracked the hue values of the restored (a) red and (b) yellow color patches at different observation distances within the Curacao scene. The first row illustrates the situation in the original underwater images. The results demonstrate that our Aquatic-GS outperforms other approaches in color correction and maintaining stable hue values.