Gaussian Difference: Find Any Change Instance in 3D Scenes

TL;DR

Gaussian Difference introduces a 4D Gaussian Splatting framework to perform 3D instance-level change detection without labeled image pairs. By embedding before/after scenes into a single $\Psi$, interpolating camera poses, segmenting instances with SAM, and tracking IDs with DEVA, it identifies change instances via ID comparison and partitions Gaussians into changed and unchanged using a learnable classification encoding to render change maps for any view. The approach achieves superior performance over C-NeRF and CYWS-3D, particularly under substantial lighting variations, while maintaining fast processing times and threshold-free operation. The authors additionally contribute a richer 3D CD dataset to support future research.

Abstract

Instance-level change detection in 3D scenes presents significant challenges, particularly in uncontrolled environments lacking labeled image pairs, consistent camera poses, or uniform lighting conditions. This paper addresses these challenges by introducing a novel approach for detecting changes in real-world scenarios. Our method leverages 4D Gaussians to embed multiple images into Gaussian distributions, enabling the rendering of two coherent image sequences. We segment each image and assign unique identifiers to instances, facilitating efficient change detection through ID comparison. Additionally, we utilize change maps and classification encodings to categorize 4D Gaussians as changed or unchanged, allowing for the rendering of comprehensive change maps from any viewpoint. Extensive experiments across various instance-level change detection datasets demonstrate that our method significantly outperforms state-of-the-art approaches like C-NERF and CYWS-3D, especially in scenarios with substantial lighting variations. Our approach offers improved detection accuracy, robustness to lighting changes, and efficient processing times, advancing the field of 3D change detection.

Figures (5)

• Figure 1: The main framework of the proposed 3D change detection method.
• Figure 2: Examples of CD results of different methods on four scenes.
• Figure 3: CD on the scenes with substantial lighting variations.
• Figure 4: Comparison of CYWS-3D, C-NeRF, and our method on different scenes. Red boxes are the predictions and blue boxes are the groundtruths.
• Figure 5: The CD results obtained by filtering Gaussian distributions with of $(\Delta x, \Delta r, \Delta s)$. The red arrow indicates the real change.