Gradient-Weighted Feature Back-Projection: A Fast Alternative to Feature Distillation in 3D Gaussian Splatting

TL;DR

This work proposes a training-free method for feature field rendering in 3D Gaussian Splatting, enabling fast and scalable embedding of high-dimensional features into 3D scenes, and achieves performance comparable to or better than training-based approaches, while significantly reducing computational cost.

Abstract

We introduce a training-free method for feature field rendering in Gaussian splatting. Our approach back-projects 2D features into pre-trained 3D Gaussians, using a weighted sum based on each Gaussian's influence in the final rendering. While most training-based feature field rendering methods excel at 2D segmentation but perform poorly at 3D segmentation without post-processing, our method achieves high-quality results in both 2D and 3D segmentation. Experimental results demonstrate that our approach is fast, scalable, and offers performance comparable to training-based methods.

Figures (4)

• Figure 1: Comparison of 2D segmentation in Feature 3DGS (F3DGS) and our method. Under each pair of image corresponding positive and negative prompts are given.
• Figure 2: Comparison of 3D segmentation in Feature 3DGS (F3DGS) and our method. Under each pair of image corresponding positive and negative prompts are given. Clearly our method produces less outliers.
• Figure 3: Qualitative result of affordance transfer. The example images are different but the same category as that of target scene.
• Figure 4: Sample result of orthogonal identity encoding. We use only about 5 frames with ground truth. The scene is trained with 30 imags. We are able to extract the occluded object as well as delete it. Note that the results are shown without post processing to remove outliers.