Gaussian Splashing: Direct Volumetric Rendering Underwater

TL;DR

Gaussian Splashing presents a CUDA-implemented underwater adaptation of 3D Gaussian Splatting that tightly integrates the Sea-thru image-formation model to enable fast geometry reconstruction and real-time novel-view rendering. By introducing learnable attenuation and backscatter parameters ($B_d$, $B_{ abla}$, $B_b$), a depth-estimation mechanism, and a backscatter-aware loss, the method achieves minute-scale training and around $140$ FPS rendering, outperforming NeRF-based underwater methods in both speed and far-field detail. The approach is validated on a new TableDB dataset and public underwater datasets, showing superior PSNR/SSIM/LPIPS and realistic distant details compared to baselines such as STNeRF, WaterSplatting, and SeaSplat. This work enables practical underwater color reconstruction and real-time visualization for robotics, exploration, and education by leveraging a physics-informed, CUDA-accelerated 3DGS pipeline.

Abstract

In underwater images, most useful features are occluded by water. The extent of the occlusion depends on imaging geometry and can vary even across a sequence of burst images. As a result, 3D reconstruction methods robust on in-air scenes, like Neural Radiance Field methods (NeRFs) or 3D Gaussian Splatting (3DGS), fail on underwater scenes. While a recent underwater adaptation of NeRFs achieved state-of-the-art results, it is impractically slow: reconstruction takes hours and its rendering rate, in frames per second (FPS), is less than 1. Here, we present a new method that takes only a few minutes for reconstruction and renders novel underwater scenes at 140 FPS. Named Gaussian Splashing, our method unifies the strengths and speed of 3DGS with an image formation model for capturing scattering, introducing innovations in the rendering and depth estimation procedures and in the 3DGS loss function. Despite the complexities of underwater adaptation, our method produces images at unparalleled speeds with superior details. Moreover, it reveals distant scene details with far greater clarity than other methods, dramatically improving reconstructed and rendered images. We demonstrate results on existing datasets and a new dataset we have collected. Additional visual results are available at: https://bgu-cs-vil.github.io/gaussiansplashingUW.github.io/ .

Figures (10)

• Figure 1: The proposed method, Gaussian Splashing, reconstructs accurate geometry for underwater scenes in minutes and achieves real-time rendering at 140 frames fer second. Here, we demonstrate an application in color reconstruction: we used the depth maps produced by our method from raw (but white balanced - WB) images as inputs to the original Sea-thru algorithm akkaynak2019, which requires an accurate depth map to estimate medium parameters. The results, shown here on images from two different distances from the new TableDB dataset we contribute, show excellent visual quality, even for distant scene details.
• Figure 2: Comparison of methods by Frames Per Second (FPS) and Peak Signal-to-Noise Ratio (PSNR). Our approach (red diamonds) achieves an impressive PSNR average of $29.11$ while maintaining high inference speeds for real-time rendering. The PSNR values were averaged over the Red Sea, Curaçao, Panama and TableDB datasets (the FPS rates are fairly consistent across datasets).
• Figure 3: Method overview: Initially, we utilize Structure from Motion (SfM) to acquire an initial point cloud and camera poses. Subsequently, we commence the optimization process to refine the model based on our underwater rendering equation and modified tile rasterization, taking those distortions into account. We evaluate backscatter every 500 steps to ensure convergence towards the accurate medium coefficients using our approach. The base figure is adapted from the original 3D Gaussian Splatting method in kerbl3Dgaussians.
• Figure 4: Visual Comparison of novel views. Row 1: an example from the Red-Sea scene. Row 3/5/7: examples from the TableDB dataset. Rows 2/4/6: zoom in on the red rectangles in Rows 1/3/5. Note that Row 7 highlights STNeRFacto's failure on unbounded scenes.
• Figure 5: Visual comparison of novel views generated by different underwater splatting methods. Evidently, our results are better than SeaSplat's, and are comparable to those of WS. That said, our method is much faster. Also, in regions far from the cameras, our method typically yields sharper results than WS.