3DGS-Drag: Dragging Gaussians for Intuitive Point-Based 3D Editing

TL;DR

3DGS-Drag tackles the challenge of intuitive, geometry-focused 3D editing by uniting deformation guidance from 3D Gaussian Splatting with diffusion-based appearance correction. The method uses handle–target point pairs to perform copy-and-paste deformations on 3D Gaussians, followed by multi-view diffusion corrections and a progressive, multi-step editing schedule to maintain consistency and detail. Key contributions include bridging deformation-based and 2D-editing-based approaches for 3D editing, introducing a progressive drag strategy with local editing masks, and achieving state-of-the-art results in motion changes, shape edits, inpainting, and content extension with practical runtimes on consumer GPUs. The approach yields efficient, high-fidelity edits across diverse real-world scenes, enabling robust 3D content manipulation suitable for creative editing and AR/VR applications.

Abstract

The transformative potential of 3D content creation has been progressively unlocked through advancements in generative models. Recently, intuitive drag editing with geometric changes has attracted significant attention in 2D editing yet remains challenging for 3D scenes. In this paper, we introduce 3DGS-Drag -- a point-based 3D editing framework that provides efficient, intuitive drag manipulation of real 3D scenes. Our approach bridges the gap between deformation-based and 2D-editing-based 3D editing methods, addressing their limitations to geometry-related content editing. We leverage two key innovations: deformation guidance utilizing 3D Gaussian Splatting for consistent geometric modifications and diffusion guidance for content correction and visual quality enhancement. A progressive editing strategy further supports aggressive 3D drag edits. Our method enables a wide range of edits, including motion change, shape adjustment, inpainting, and content extension. Experimental results demonstrate the effectiveness of 3DGS-Drag in various scenes, achieving state-of-the-art performance in geometry-related 3D content editing. Notably, the editing is efficient, taking 10 to 20 minutes on a single RTX 4090 GPU.

Figures (14)

• Figure 1: Our proposed 3DGS-Drag framework enables high-quality 3D drag editing: Users only need to input 3D handle points (circle) and target points (triangle). Our method precisely moves the handle points to match the target points while preserving the overall content and details.
• Figure 2: Overview of 3DGS-Drag : Given a trained 3D Gaussian splatting model and the dataset, we use the multi-step editing scheduler to calculate the intermediate handle points $p'_h(i)$ and target points $p'_t(i)$ for step $i$. In each step, we first deform the 3D Gaussians using handle points and target points. Then, we render the image for each view and correct it with a diffusion model. The final corrected images will be used to train 3D Gaussians to improve quality. The diffusion model is fine-tuned with LoRA for more consistent edits.
• Figure 3: Multi-view consistent 2D edits: With the deformed rendering as input, the fine-tuned diffusion model can perform multi-view consistent edits, and the artifacts and incorrect parts (shoes) are fixed.
• Figure 4: Intermediate dragging steps and tracked mask: Our method conducts progressive editing toward the target point. The dragged Gaussians are tracked to achieve aggressive edits.
• Figure 5: Qualitative results in various scenes: Our method can handle complex scenes and generate highly detailed results. With a simple drag input, 3DGS-Drag can identify the 3D context and perform edits like moving objects, inpainting the background, adjusting appearance, modifying object shape, and adjusting motion. The orange bounding boxes highlight the modified regions.