Rethinking Pose Refinement in 3D Gaussian Splatting under Pose Prior and Geometric Uncertainty

Abstract

3D Gaussian Splatting (3DGS) has recently emerged as a powerful scene representation and is increasingly used for visual localization and pose refinement. However, despite its high-quality differentiable rendering, the robustness of 3DGS-based pose refinement remains highly sensitive to both the initial camera pose and the reconstructed geometry. In this work, we take a closer look at these limitations and identify two major sources of uncertainty: (i) pose prior uncertainty, which often arises from regression or retrieval models that output a single deterministic estimate, and (ii) geometric uncertainty, caused by imperfections in the 3DGS reconstruction that propagate errors into PnP solvers. Such uncertainties can distort reprojection geometry and destabilize optimization, even when the rendered appearance still looks plausible. To address these uncertainties, we introduce a relocalization framework that combines Monte Carlo pose sampling with Fisher Information-based PnP optimization. Our method explicitly accounts for both pose and geometric uncertainty and requires no retraining or additional supervision. Across diverse indoor and outdoor benchmarks, our approach consistently improves localization accuracy and significantly increases stability under pose and depth noise.

Figures (11)

• Figure 1: UGS-Loc. Our proposed framework refines camera pose by incorporating pose prior uncertainty via Monte Carlo sampling and geometric uncertainty via Uncertainty fields, achieving robust localization without retraining.
• Figure 2: Comparison between Previous 3DGS-based Pose Refinement and Our Proposed UGS-Loc. (a) Existing pipelines refine a single deterministic pose prior using rendered depth, which makes them vulnerable to inaccurate priors and geometric errors—leading to bad correspondences and unstable refinement. (b) Our method instead applies Monte Carlo pose sampling and geometric uncertainty from Fisher Information to guide correspondences and focus refinement on reliable regions, achieving robust pose estimation even under poor pose priors and uncertain geometry.
• Figure 3: Distributions of Error–Confidence and Error–Uncertainty. We visualize the relationship between translation error and the aggregated matcher confidence (a,c) or aggregated depth uncertainty (b,d) for frames from the Kings (blue) and Hospital (orange) scenes of the Cambridge dataset. For each frame, the x-axis value represents the sum of confidence or uncertainty across all its 2D–2D correspondences. The green trend line illustrates that higher confidence and lower uncertainty consistently correlate with smaller localization errors.
• Figure 4: Effect of Uncertainty-based PnP. We visualize the rendered uncertainty map together with 2D–2D correspondences projected onto the synthesized view. Green and red points denote inlier and outlier points, respectively. When applying our uncertainty-guided PnP, correspondences located in high-uncertainty regions are naturally suppressed, yielding more reliable inliers and improved pose estimate results.
• Figure 5: Visualization of Localization Quality on the 7-Scenes dataset. Each triplet of images compares the ground-truth view (top-left) with the view rendered from (i) the input pose prior, (ii) the baseline GS-CPR refinement, and (iii) our UGS-Loc refinement (bottom-right). A closer alignment along the diagonal boundary indicates better pose accuracy.