Self-supervised Physics-Informed Manipulation of Deformable Linear Objects with Non-negligible Dynamics

TL;DR

SPiD introduces a physics-informed, self-supervised framework for dynamic manipulation of deformable linear objects by coupling a damping-enhanced mass–spring rope model with differentiable physics-based system identification and augmented self-supervised learning. A neural controller is trained end-to-end against a task loss, with curriculum-based identification, data augmentation, and a self-supervised DAgger mechanism to mitigate distribution shifts, plus domain randomization for robustness. The approach is validated on rope stabilization and trajectory tracking in both simulation and real hardware, achieving fast, smooth stabilization and robust sim-to-real generalization even with markerless perception. The work advances data efficiency and robustness in fast, dynamic DLO manipulation, paving the way for scalable deployment in real-world settings where perception and dynamics are challenging. Limitations include applicability to one-dimensional topologies, with future work targeting cloth and other 2D/3D deformable objects and broader tasks.

Abstract

We address dynamic manipulation of deformable linear objects by presenting SPiD, a physics-informed self-supervised learning framework that couples an accurate deformable object model with an augmented self-supervised training strategy. On the modeling side, we extend a mass-spring model to more accurately capture object dynamics while remaining lightweight enough for high-throughput rollouts during self-supervised learning. On the learning side, we train a neural controller using a task-oriented cost, enabling end-to-end optimization through interaction with the differentiable object model. In addition, we propose a self-supervised DAgger variant that detects distribution shift during deployment and performs offline self-correction to further enhance robustness without expert supervision. We evaluate our method primarily on the rope stabilization task, where a robot must bring a swinging rope to rest as quickly and smoothly as possible. Extensive experiments in both simulation and the real world demonstrate that the proposed controller achieves fast and smooth rope stabilization, generalizing across unseen initial states, rope lengths, masses, non-uniform mass distributions, and external disturbances. Additionally, we develop an affordable markerless rope perception method and demonstrate that our controller maintains performance with noisy and low-frequency state updates. Furthermore, we demonstrate the generality of the framework by extending it to the rope trajectory tracking task. Overall, SPiD offers a data-efficient, robust, and physically grounded framework for dynamic manipulation of deformable linear objects, featuring strong sim-to-real generalization.

Figures (14)

• Figure 1: Dynamic manipulation for rope stabilization. Comparison between our controller and the passive case (where the robot arm holds its position). Each subfigure shows the rope motion at the current time, with 12 ghosted frames traced backward at 37.5 ms intervals—the fainter the trace, the earlier the time. Our controller actively computes actions that minimize the rope’s energy in real time, achieving superior performance, whereas the passive case relies solely on air resistance to dissipate energy.
• Figure 2: The data flow of the SPiD framework.
• Figure 3: The DLO model. The model consists of $N{+}1$ mass points connected by linear springs/dampers, bending springs/dampers, and torsional springs. Additionally, air drag and gravity are incorporated into dynamics.
• Figure 4: Model validation. Rope tip trajectories of the reference rope (Mujoco or real), our model, and the baseline model under three unseen initial states. Our model can predict future states with higher accuracy than the baseline model.
• Figure 5: Rope's motion sequence of Case 3 (Real). The motion trajectory of the rope over the first 3 s after release. Each subplot corresponds to a 0.25 s time window and shows three uniformly sampled rope configurations. Lighter colors indicate earlier time instances.

Theorems & Definitions (2)

• Proposition 1
• proof