CoMapGS: Covisibility Map-based Gaussian Splatting for Sparse Novel View Synthesis

TL;DR

CoMapGS addresses region-wise uncertainty in sparse novel view synthesis by introducing covisibility maps as a core supervisory signal. It enhances initial point clouds with triangulated dense correspondences and monocular-depth points and employs a proximity classifier to apply covisibility-weighted supervision across both multiview and mono-view regions, encapsulated in an augmented 3D Gaussian Splatting objective. The method demonstrates consistent, state-of-the-art improvements on LLFF and Mip-NeRF 360 datasets, particularly in underrepresented high-uncertainty areas, and remains compatible with existing sparse-view frameworks. This approach offers practical gains in texture fidelity and geometric consistency for real-world sparse capture scenarios, including outdoor scenes with varying covisibility.

Abstract

We propose Covisibility Map-based Gaussian Splatting (CoMapGS), designed to recover underrepresented sparse regions in sparse novel view synthesis. CoMapGS addresses both high- and low-uncertainty regions by constructing covisibility maps, enhancing initial point clouds, and applying uncertainty-aware weighted supervision using a proximity classifier. Our contributions are threefold: (1) CoMapGS reframes novel view synthesis by leveraging covisibility maps as a core component to address region-specific uncertainty; (2) Enhanced initial point clouds for both low- and high-uncertainty regions compensate for sparse COLMAP-derived point clouds, improving reconstruction quality and benefiting few-shot 3DGS methods; (3) Adaptive supervision with covisibility-score-based weighting and proximity classification achieves consistent performance gains across scenes with varying sparsity scores derived from covisibility maps. Experimental results demonstrate that CoMapGS outperforms state-of-the-art methods on datasets including Mip-NeRF 360 and LLFF.

Figures (11)

• Figure 1: Main idea. Conventional sparse view synthesis methods using 3DGS Zhang_2024_ECCV_CorGS prioritize frequently captured regions, shown as the brightest areas in the covisibility map, resulting in better reconstruction in these regions but missing details in sparsely captured areas. By using a covisibility map, we guide sparse 3DGS methods to focus on underrepresented regions, enhancing sparse novel view synthesis.
• Figure 2: Overview of the proposed CoMapGS. CoMapGS leverages existing modules, including Structure-from-Motion (SfM), monocular depth, and dense correspondence prediction, to produce preliminary outputs. Covisibility maps are generated and used alongside module outputs to enhance initial point clouds, which initialize Gaussians for CoMapGS. During training, covisibility map-based weighted supervision with the proximity classifier applies varied strengths to Gaussians based on region-specific covisibility.
• Figure 3: Examples of enhanced initial PCLs from the Fern scene in the LLFF dataset. (a) Initial SfM points are first (b) updated through triangulation using dense correspondences and COLMAP camera poses, and then (c) fully refined to include mono-view regions (mono-view updates are colored in red for visualization).
• Figure 4: Alongside (a) the conventional reconstruction loss $\mathcal{L}1$, we add (b) a weighted proximity loss $\mathcal{L}_{p}$ that applies classifier results with assigned weights based on Gaussian projections onto the covisibility map and their positions relative to the frustum.
• Figure 5: Qualitative comparison on 3 training views from the LLFF dataset. We compare our results with FSGS Zhu_2024_ECCV_FSGS and CoR-GS Zhang_2024_ECCV_CorGS, demonstrating that our method achieves superior texture synthesis in novel views, particularly in high-uncertainty mono-view regions that appear only once in the training set.