SketchRodGS: Sketch-based Extraction of Slender Geometries for Animating Gaussian Splatting Scenes

TL;DR

Slender structures in Gaussian splatting lack robust connectivity for physics-based animation. The paper introduces a screen-space shortest-path approach solved by dynamic programming to extract a 3D polyline guided by a user sketch, embedding the slender part for elastic rod simulation. Key contributions include robust connectivity extraction from noisy Gaussian primitives, occlusion handling, and real-time performance on in-the-wild data, with an end-to-end pipeline that integrates skinning and discrete elastic rod simulation. This enables interactive, deformation-aware content creation directly from Gaussian splatting representations and lays groundwork for extending to generalized cylinders and surface meshes.

Abstract

Physics simulation of slender elastic objects often requires discretization as a polyline. However, constructing a polyline from Gaussian splatting is challenging as Gaussian splatting lacks connectivity information and the configuration of Gaussian primitives contains much noise. This paper presents a method to extract a polyline representation of the slender part of the objects in a Gaussian splatting scene from the user's sketching input. Our method robustly constructs a polyline mesh that represents the slender parts using the screen-space shortest path analysis that can be efficiently solved using dynamic programming. We demonstrate the effectiveness of our approach in several in-the-wild examples.

Figures (4)

• Figure 1: Workflow. (a) The user inputs a stroke $\mathbf{s}$ in a viewpoint, and the index of the Gaussian primitive for each pixel $\mathcal{K}$ is computed for that viewpoint. (b) The shortest path connecting pixels is computed by dynamic programming. (c) We smooth and resample the polyline connecting the center of Gaussian primitives to obtain a polyline for simulation. We segment the primitives inside the cylinder with radius $R$. (d) We animate the Gaussian primitives using the linear blend skinning based on the elastic rod simulation on the polyline.
• Figure 2: Left: Naively connecting the shortest path on the screen may result in discontinuities when there are occlusions. Right: to overcome this issue, we do not connect pixels with a large depth gap. In addition, we skip the vertices in the stroke if there is a depth gap.
• Figure 3: Examples of manipulating slender objects using our system. Each pair images shows the undeformed object (inset) and resulting real-time deformation by pulling the 3D polyline.
• Figure 4: Comparison against a baseline using 3D k-nearest neighbor path finding. Given a stroke (a), the naïve method stops working in the middle due to noisy, highly elongated Gaussian primitives (b). On the other hand, our screen space approach finds the path covering the entire stroke (c).