3D-UIR: 3D Gaussian for Underwater 3D Scene Reconstruction via Physics Based Appearance-Medium Decoupling

TL;DR

This work tackles underwater novel view synthesis by addressing light-medium interactions that cause artifacts and view-dependent color distortions. It introduces a physics-based framework built on 3D Gaussian Splatting that decouples object appearance from the water medium via Underwater Appearance Modeling and Scatter Medium Modeling, integrated through an Underwater Image Formation Model. Depth-guided Regularization Optimization, including a Gaussian flattening constraint, and pseudo-depth supervision from DepthAnythingV2, improve geometric fidelity and depth consistency while enabling water-removed scene restoration. The approach demonstrates significant improvements in rendering quality and restoration accuracy across multiple underwater datasets, while maintaining real-time performance. This combination of explicit medium modeling, appearance disentanglement, and depth-guided optimization advances underwater 3D reconstruction and view synthesis with practical implications for marine mapping and autonomous underwater operations.

Abstract

Novel view synthesis for underwater scene reconstruction presents unique challenges due to complex light-media interactions. Optical scattering and absorption in water body bring inhomogeneous medium attenuation interference that disrupts conventional volume rendering assumptions of uniform propagation medium. While 3D Gaussian Splatting (3DGS) offers real-time rendering capabilities, it struggles with underwater inhomogeneous environments where scattering media introduce artifacts and inconsistent appearance. In this study, we propose a physics-based framework that disentangles object appearance from water medium effects through tailored Gaussian modeling. Our approach introduces appearance embeddings, which are explicit medium representations for backscatter and attenuation, enhancing scene consistency. In addition, we propose a distance-guided optimization strategy that leverages pseudo-depth maps as supervision with depth regularization and scale penalty terms to improve geometric fidelity. By integrating the proposed appearance and medium modeling components via an underwater imaging model, our approach achieves both high-quality novel view synthesis and physically accurate scene restoration. Experiments demonstrate our significant improvements in rendering quality and restoration accuracy over existing methods. The project page is available at https://bilityniu.github.io/3D-UIR.

Figures (9)

• Figure 1: Optical scattering and absorption in underwater environments present unique challenges for novel view synthesis. The standard volume rendering equation inadequately models participating media with suspended particles, resulting in the incorrect reconstruction of volumetric water as the floating artifacts in 3D representation (left). Additionally, light source directionality and viewing angles cause attenuation variations that result in inconsistent scene appearance across viewpoints (right). 3DGS 3DGS_2023 lacks proper modeling of scattering media, causing water column effects on scene surfaces and resulting in collapsed depth representations. Existing scattering Novel View Synthesis (NVS) methods, e.g., SeaThru-NeRF SeaThru-NeRF_2023_CVPR, fail to account for dynamic photometric variations, which introduces water float artifacts (highlighted in yellow boxes). In contrast, our approach effectively models the participating medium to render photorealistic novel views with accurate scene representation, yielding more consistent scene rendering across novel viewpoints and effective elimination of underwater artifacts.
• Figure 2: Overview of the proposed method. Our method disentangles object appearance from water medium effects through specialized Gaussian modeling. Underwater Appearance Modeling (UAM) branch incorporates appearance features and embeddings to handle view consistency challenges. Scatter Medium Modeling (SMM) branch separately models backscatter and attenuation. Depth-guided Regularization Optimization (DRO) uses pseudo-depth maps to improve parameter estimation. All components integrate through a physics-based underwater image formation model during differentiable rasterization.
• Figure 3: The top illustrates an overview of the proposed Scatter Medium Modeling framework, which decomposes underwater medium effects based on a physical imaging model. The bottom presents the detailed architectures of the Backscatter Estimation Module (BEM) and the Attenuation Estimation Module (AEM), respectively. The BEM computes the backscatter component, while the AEM estimates the attenuation coefficients of the direct component, effectively modeling the complex light transport in underwater environments.
• Figure 4: Qualitative comparison of underwater NVS tasks on real-world and simulated datasets. We visualize the rendered depth maps and zoomed-in scene details. The proposed method outperforms other approaches in capturing scene details and maintaining scene consistency, whereas other methods exhibit floating artifacts and inaccurate depth estimation (Zoom-in for best view).
• Figure 5: Visualization of novel view restoration on the Seathru-NeRF Dataset SeaThru-NeRF_2023_CVPR. Our method achieves superior restoration quality with impressive visibility and scene consistency compared to other underwater NVS methods.