PolarGS: Polarimetric Cues for Ambiguity-Free Gaussian Splatting with Accurate Geometry Recovery

TL;DR

<3-5 sentence high-level summary> PolarGS addresses geometric reconstruction failures of RGB-based 3D Gaussian Splatting in photometrically ambiguous regions by injecting polarization cues as optical priors. It introduces two modules: polarization-guided photometric correction to stabilize reflective Gaussians using DoLP-based localization, PRI, and Color Refinement Maps; and a polarization-enhanced Gaussian densification that leverages AoLP/DoLP within a PatchMatch framework to recover missing geometry. The method is framework-agnostic and demonstrates superior geometric accuracy and mesh completeness on real and synthetic datasets, outperforming state-of-the-art methods and showing strong plug-and-play compatibility. This work highlights the value of physics-informed polarization priors for robust, high-fidelity 3D reconstruction with Gaussian representations.

Abstract

Recent advances in surface reconstruction for 3D Gaussian Splatting (3DGS) have enabled remarkable geometric accuracy. However, their performance degrades in photometrically ambiguous regions such as reflective and textureless surfaces, where unreliable cues disrupt photometric consistency and hinder accurate geometry estimation. Reflected light is often partially polarized in a manner that reveals surface orientation, making polarization an optic complement to photometric cues in resolving such ambiguities. Therefore, we propose PolarGS, an optics-aware extension of RGB-based 3DGS that leverages polarization as an optical prior to resolve photometric ambiguities and enhance reconstruction accuracy. Specifically, we introduce two complementary modules: a polarization-guided photometric correction strategy, which ensures photometric consistency by identifying reflective regions via the Degree of Linear Polarization (DoLP) and refining reflective Gaussians with Color Refinement Maps; and a polarization-enhanced Gaussian densification mechanism for textureless area geometry recovery, which integrates both Angle and Degree of Linear Polarization (A/DoLP) into a PatchMatch-based depth completion process. This enables the back-projection and fusion of new Gaussians, leading to more complete reconstruction. PolarGS is framework-agnostic and achieves superior geometric accuracy compared to state-of-the-art methods.

Figures (10)

• Figure 1: Comparison on NeISF dataset li2024neisf between standalone 3DGS kerbl20233d, PGSR chen2024pgsr and PGSR plugged in with our method. Our method not only produces faithful geometry in photometrically ambiguous regions, but can also serve as a plug-and-play module to improve 3DGS-based surface reconstruction pipelines.
• Figure 2: Overview of PolarGS. First, we preprocess polarized images to extract the $s_0$ vector to initialize a Gaussian point cloud and compute the Polarimetric Reference Intensity (PRI). Based on PRI, we generate Color Refinement Maps (CRMs, i.e. $I_{\text{diff}}$ and $I_{\text{chro}}$). During polar-guided photometric correction stage, we localize reflective regions and optimize reflective Gaussians using reflective-aware loss function supervised by CRMs. During polar-enhanced Gaussian densification, we integrate A/DoLP cues into the PatchMatch algorithm to predict depth maps, which are then back-projected into 3D space to generate new Gaussians. Finally, we apply TSDF fusion to rendered depth maps by $\alpha$-blending for mesh extraction. It should be noted that PolarGS is compatible with various mesh extraction strategies and can be integrated into existing pipelines.
• Figure 3: CRMs and Rendered Image. Guided by the CRMs in (b) and (c), the specular highlights in the original input (a), are effectively eliminated in the 3DGS-rendered image (d). Blue and red rectangles denote specular and overexposed regions, respectively.
• Figure 4: We extend vanilla PatchMatch with A/DoLPs and regularize it via azimuth consistency $S_a$ and alignment score $S_{nd}$, yielding accurate estimates of $d^{opt}$ and $\mathbf{n}^{opt}$, which are further used to back-project and fuse new Gaussians.
• Figure 5: Visual comparison of the qualitative evaluation with state-of-the-art methods on real captured scenes.