Plenodium: UnderWater 3D Scene Reconstruction with Plenoptic Medium Representation

TL;DR

Plenodium addresses underwater 3D reconstruction by jointly modeling objects and participating media with a plenoptic medium encoded by spherical harmonics. It introduces pseudo-depth Gaussian complementation to robustly initialize 3D Gaussian splats and a depth-ranking loss to enforce depth order, plus a multi-term photometric/structural loss for stable optimization. On real SeaThru-NeRF data and synthetic degraded scenes, Plenodium achieves higher PSNR/SSIM and substantially faster rendering than prior methods, demonstrating improved robustness to scattering and degradation. The approach enables more faithful underwater scene reconstruction and restoration, with code and data publicly available.

Abstract

We present Plenodium (plenoptic medium), an effective and efficient 3D representation framework capable of jointly modeling both objects and participating media. In contrast to existing medium representations that rely solely on view-dependent modeling, our novel plenoptic medium representation incorporates both directional and positional information through spherical harmonics encoding, enabling highly accurate underwater scene reconstruction. To address the initialization challenge in degraded underwater environments, we propose the pseudo-depth Gaussian complementation to augment COLMAP-derived point clouds with robust depth priors. In addition, a depth ranking regularized loss is developed to optimize the geometry of the scene and improve the ordinal consistency of the depth maps. Extensive experiments on real-world underwater datasets demonstrate that our method achieves significant improvements in 3D reconstruction. Furthermore, we conduct a simulated dataset with ground truth and the controllable scattering medium to demonstrate the restoration capability of our method in underwater scenarios. Our code and dataset are available at https://plenodium.github.io/.

Figures (7)

• Figure 1: Comparison of Plenodium, WaterSplatting watersplatting, and SeaThru-NeRF Seathru-nerf on reconstruction and restoration performance (PSRN, SSIM), as well as efficiency (FPS, training time).
• Figure 2: Overview of our Plenodium. We first employ the pseudo-depth Gaussian complementation to enrich the primitives initialized by COLMAP. Then we utilize the plenoptic medium representation to estimate the medium parameter and render the underwater images following Eqn. \ref{['eq: render_color']}. Our network is optimized with the loss function in Eqn. \ref{['eq: loss']}, including a new depth ranking regularized loss.
• Figure 3: Rendering performance comparison of our Plenodium against existing methods on the "Curaçao" and "IUI3 Red Sea" scenes. Rendered images and depths are presented for comparison. The pseudo-depth for ground truth is estimated using the Depth Any Model DepthanythingDepthanythingv2 for reference purposes. Compared to competing methods, Plenodium enhances clarity for medium and distant objects, as highlighted in the red boxes, while simultaneously yielding more precise depth maps.
• Figure 4: Restoration performance comparison of our Plenodium against existing methods on the "JapaneseGarden Red Sea" and "Panama" scenes. As shown in the red boxes, the proposed Plenodium generates results with more reasonable exposure and accurate colors.
• Figure 5: Restoration performance comparison of our Plenodium against existing methods on "Beach" and "Street" scenes from our simulated dataset. Quantitative evaluations, including PSNR and SSIM metrics for each image, are beneath the corresponding figures. Among these methods, Plenodium produces results with more coherent textural details and superior color accuracy.