COB-GS: Clear Object Boundaries in 3DGS Segmentation Based on Boundary-Adaptive Gaussian Splitting

TL;DR

This work tackles blurry boundaries in 3D Gaussian Splatting (3DGS) segmentation by proposing COB-GS, a framework that jointly optimizes semantic masks and scene textures. It introduces boundary-adaptive Gaussian splitting driven by semantic gradient statistics and a boundary-guided texture restoration, complemented by a two-stage, text-prompt–assisted mask generation to robustly supervise the 3DGS refinement. Empirical results on NVOS and open-vocabulary datasets show substantial improvements in segmentation accuracy (mIoU around $92.2\%$) and boundary quality while preserving high visual fidelity, demonstrating the practical potential of coupling semantics and appearance in explicit 3D scene representations. The approach supports open vocabulary segmentation and scalable multi-object, multi-granularity segmentation, with future work aimed at removing floating artifacts and further reducing computation.

Abstract

Accurate object segmentation is crucial for high-quality scene understanding in the 3D vision domain. However, 3D segmentation based on 3D Gaussian Splatting (3DGS) struggles with accurately delineating object boundaries, as Gaussian primitives often span across object edges due to their inherent volume and the lack of semantic guidance during training. In order to tackle these challenges, we introduce Clear Object Boundaries for 3DGS Segmentation (COB-GS), which aims to improve segmentation accuracy by clearly delineating blurry boundaries of interwoven Gaussian primitives within the scene. Unlike existing approaches that remove ambiguous Gaussians and sacrifice visual quality, COB-GS, as a 3DGS refinement method, jointly optimizes semantic and visual information, allowing the two different levels to cooperate with each other effectively. Specifically, for the semantic guidance, we introduce a boundary-adaptive Gaussian splitting technique that leverages semantic gradient statistics to identify and split ambiguous Gaussians, aligning them closely with object boundaries. For the visual optimization, we rectify the degraded suboptimal texture of the 3DGS scene, particularly along the refined boundary structures. Experimental results show that COB-GS substantially improves segmentation accuracy and robustness against inaccurate masks from pre-trained model, yielding clear boundaries while preserving high visual quality. Code is available at https://github.com/ZestfulJX/COB-GS.

Figures (10)

• Figure 1: Problems on 3DGS segmentation. Our task is to achieve high-quality 3D segmentation on 3DGS, including foreground and background. Take foreground as an example: (a) unclear segmentation result of existing methods; (b) blurry boundary Gaussians; (c) segmentation results from joint optimization; (d) tiny ambiguous Gaussians due to incorrect masks from pre-trained models; (e) final segmentation results after improving robustness from COB-GS.
• Figure 2: Pipeline of our clear object boundary segmentation method for 3DGS. On the left, we present our two-stage mask generation method, which utilizes SAM2 perform mask prediction on image sequences based on text prompt to obtain masks for regions of interest. Images and masks serve as supervision for 3DGS refinement. On the right, for the reconstructed 3DGS scene, we jointly and alternately optimize the mask and texture. For the mask optimization, boundary-adaptive Gaussian splitting is performed to refine boundary structure.
• Figure 3: Supervision based solely on texture results in large Gaussians due to the similarity in textures between objects. However, object-level mask supervision facilitates the differentiation of object edges. This allows 3D Gaussians to split along object edges, while also guiding the correct restoration of scene textures.
• Figure 4: Visualization of different processing phases. (a) Optimizing the mask labels without Gaussian splitting results in unclear boundary segmentation. (b) Jointly optimizing masks and textures with boundary-adaptive Gaussian splitting effectively reduces large ambiguous Gaussians while still leaving tiny ones at the boundaries. (c) Extract and refine tiny ambiguous boundary Gaussians obtained by non-convergent splitting. These tiny Gaussians have little impact on visual quality but affect boundary clarity in 3D segmentation.
• Figure 5: Qualitative result of single-object segmentation. The results show that our method segments the boundaries of the object more clearly, without blurred Gaussians, and the background is cleaner after the object removal.