TIBR4D: Tracing-Guided Iterative Boundary Refinement for Efficient 4D Gaussian Segmentation

TL;DR

This paper presents an efficient learning-free 4D Gaussian segmentation framework that lifts video segmentation masks to 4D spaces, whose core is a two-stage iterative boundary refinement, TIBR4D.

Abstract

Object-level segmentation in dynamic 4D Gaussian scenes remains challenging due to complex motion, occlusions, and ambiguous boundaries. In this paper, we present an efficient learning-free 4D Gaussian segmentation framework that lifts video segmentation masks to 4D spaces, whose core is a two-stage iterative boundary refinement, TIBR4D. The first stage is an Iterative Gaussian Instance Tracing (IGIT) at the temporal segment level. It progressively refines Gaussian-to-instance probabilities through iterative tracing, and extracts corresponding Gaussian point clouds that better handle occlusions and preserve completeness of object structures compared to existing one-shot threshold-based methods. The second stage is a frame-wise Gaussian Rendering Range Control (RCC) via suppressing highly uncertain Gaussians near object boundaries while retaining their core contributions for more accurate boundaries. Furthermore, a temporal segmentation merging strategy is proposed for IGIT to balance identity consistency and dynamic awareness. Longer segments enforce stronger multi-frame constraints for stable identities, while shorter segments allow identity changes to be captured promptly. Experiments on HyperNeRF and Neu3D demonstrate that our method produces accurate object Gaussian point clouds with clearer boundaries and higher efficiency compared to SOTA methods.

Figures (7)

• Figure 1: Comparison with prior works SA4D ji2024segment and SADG li2024sadg. Previous methods often suffer from overflow at object boundaries. Benefiting from two convergent iterative stages, our method enables fast extraction of target objects while effectively suppressing floating Gaussians and boundary leakage, resulting in more accurate segmentation boundaries.
• Figure 2: Overview. Our learning-free object segmentation framework of 4D Gaussian scenes consists of two iterative stages: iterative Gaussian instance tracing (Sec. \ref{['sec:IGIT']}) with temporal segmentation (Sec. \ref{['sec:TSM']}), and frame-wise Gaussian rendering range control (Sec. \ref{['sec:RRC']}). In the first stage, we iteratively perform Gaussian Instance Tracing within temporal segments to remove erroneous Gaussians and merge adjacent segments when their extracted Gaussian point clouds become consistent. In the second stage, we progressively truncate the outer regions of each projected 2D Gaussian and retain only its most reliable central contribution. After these two convergent stages, we obtain a clean Gaussian point cloud for the target object.
• Figure 3: Qualitative comparison of our method’s performance against SA4D ji2024segment and SADG li2024sadg on the HyperNeRF park2021hypernerf dataset.
• Figure 4: Qualitative comparison of our method’s performance against SA4D ji2024segment and SADG li2024sadg on the Neu3D li2022neural dataset.
• Figure 4: Ablation study.We evaluate our method under different configurations on cut-lemon1 and chickchicken.