DLO-Splatting: Tracking Deformable Linear Objects Using 3D Gaussian Splatting

TL;DR

DLO-Splatting addresses the challenge of tracking 3D shapes of Deformable Linear Objects (DLOs) under complex topologies and occlusions by fusing a training-free prediction step based on Position-Based Dynamics with a 3D Gaussian Splatting rendering update. The method forms a prediction-update framework, where the predicted rope state $\hat{\bm{X}}^{t+1}$ is refined through a rendering loss $\mathcal{L}_{obs}$ computed against multi-view RGB observations, yielding $\hat{\bm{X}}^{t+1} = \bm{X}^{t+1}_{GS} = \bm{X}^{t+1}_{PBD} + \Delta \bm{X}^{t+1}$ and optimized via SGD. Key contributions include replacing learned dynamics with PBD, introducing a Gaussian Splatting rendering update for state refinement, and demonstrating qualitative tracking improvements in knot-tying scenarios. The findings suggest the approach can handle visually tricky deformations that hinder vision-only methods, though challenges remain in final topology recovery, occlusions, and update-rate limitations. Future work aims to enhance physical modeling fidelity, increase update frequency, and integrate scene-level splatting and improved DLO instance segmentation for multi-object settings.

Abstract

This work presents DLO-Splatting, an algorithm for estimating the 3D shape of Deformable Linear Objects (DLOs) from multi-view RGB images and gripper state information through prediction-update filtering. The DLO-Splatting algorithm uses a position-based dynamics model with shape smoothness and rigidity dampening corrections to predict the object shape. Optimization with a 3D Gaussian Splatting-based rendering loss iteratively renders and refines the prediction to align it with the visual observations in the update step. Initial experiments demonstrate promising results in a knot tying scenario, which is challenging for existing vision-only methods.

Figures (2)

• Figure 1: The DLO-Splatting Algorithm. The DLO-Splatting algorithm estimates the 3D state of a DLO using a prediction-update framework akin to Bayesian filtering. The DLO state is predicted using position-based dynamics and is iteratively updated using 3D Gaussian Splatting-based rendering.
• Figure 2: Qualitative Results. The DLO-Splatting algorithm is compared to TrackDLO on qualitative tracking during a cross move commonly performed in knot tying. During this move, one tip of the DLO is moved through a loop to create an additional crossing in the topology. After eight seconds, both DLO-Splatting and TrackDLO failed to estimate the correct topology of the DLO, however DLO-Splatting succeeds in tracking the grasped tip.