CoRe-GS: Coarse-to-Refined Gaussian Splatting with Semantic Object Focus

TL;DR

CoRe-GS drastically reduces training time compared to full semantic GS while improving POI reconstruction quality and mitigating floaters, and introduces a color-based filtering mechanism that removes inconsistent Gaussians without requiring mask rasterization.

Abstract

Fast and efficient 3D reconstruction is essential for time-critical robotic applications such as tele-guidance and disaster response, where operators must rapidly analyze specific points of interest (POIs). Existing semantic Gaussian Splatting (GS) approaches optimize the entire scene uniformly, incurring substantial computational cost even when only a small subset of the scene is operationally relevant. We propose CoRe-GS, a coarse-to-refine GS framework that enables task-driven POI-focused optimization. Our method first produces a segmentation-ready GS representation using a lightweight late-stage semantic refinement. Subsequently, only Gaussians associated with the selected POI are further optimized, reducing unnecessary background computation. To mitigate segmentation-induced outliers (floaters) during selective refinement, we introduce a color-based filtering mechanism that removes inconsistent Gaussians without requiring mask rasterization. We evaluate robustness multiple datasets. On LERF-Mask, our segmentation-ready representation achieves competitive mIoU using tremendously fewer optimization steps. Across synthetic and real-world datasets (NeRDS360, SCRREAM, Tanks and Temples), CoRe-GS drastically reduces training time compared to full semantic GS while improving POI reconstruction quality and mitigating floaters. These results demonstrate that task-aware selective refinement enables faster and higher-quality scene reconstruction tailored to robotic operational needs.

Figures (7)

• Figure 1: We extract the injured person as poi more consistently and at significantly lower runtime (right) compared to prior work lyu_gaga_2024ye_gaussian_2024. Notably, mask association in GAGA lyu_gaga_2024 fails, whereas our method successfully extracts the injured person and the final optimized poi.$^{1}$
• Figure 2: CoRe-GS. First, we produce a coarse gs representation including a lightweight semantic optimization step (left). The poi is extracted based on the semantic segmentation associations created in the previous step (center). Lastly, poi refinement leverages our periodic scene filtering (right).
• Figure 3: Training time vs psnr and ssim on NeRDS 360. We highlight, a higher poi isolation performance by tremendously lower runtime.
• Figure 4: poi isolation. We compare gg ye_gaussian_2024, GAGA lyu_gaga_2024, SAGD hu_sagd_2024 an ours on NeRDS360.
• Figure 5: Filtering on SF_6thAndMission_medium6. Our approach without filtering shows outliers (right, bottom), while filtering leads to a clean result (left, bottom). Different filtering iterations can alread reduce outliers (top).