Rapidly Learning Soft Robot Control via Implicit Time-Stepping

TL;DR

The paper tackles the slow pace of soft-robot policy learning by leveraging a fully implicit time-stepping soft-body simulator, DisMech, together with a delta natural curvature control formulation. It demonstrates that DisMech can match Elastica's dynamics while delivering substantial speedups in training, especially under high-contact scenarios, and shows a favorable sim-to-sim transfer profile. The study provides extensive task-based comparisons across four soft-manipulator tasks, highlighting that implicit time-stepping enables rapid data collection without sacrificing accuracy. By introducing a practical delta curvature control scheme and releasing a benchmarking setup, the work offers a scalable path for rapid soft-robot policy development and evaluation.

Abstract

With the explosive growth of rigid-body simulators, policy learning in simulation has become the de facto standard for most rigid morphologies. In contrast, soft robotic simulation frameworks remain scarce and are seldom adopted by the soft robotics community. This gap stems partly from the lack of easy-to-use, general-purpose frameworks and partly from the high computational cost of accurately simulating continuum mechanics, which often renders policy learning infeasible. In this work, we demonstrate that rapid soft robot policy learning is indeed achievable via implicit time-stepping. Our simulator of choice, DisMech, is a general-purpose, fully implicit soft-body simulator capable of handling both soft dynamics and frictional contact. We further introduce delta natural curvature control, a method analogous to delta joint position control in rigid manipulators, providing an intuitive and effective means of enacting control for soft robot learning. To highlight the benefits of implicit time-stepping and delta curvature control, we conduct extensive comparisons across four diverse soft manipulator tasks against one of the most widely used soft-body frameworks, Elastica. With implicit time-stepping, parallel stepping of 500 environments achieves up to 6x faster speeds for non-contact cases and up to 40x faster for contact-rich scenarios. Finally, a comprehensive sim-to-sim gap evaluation--training policies in one simulator and evaluating them in another--demonstrates that implicit time-stepping provides a rare free lunch: dramatic speedups achieved without sacrificing accuracy.

Figures (3)

• Figure 1: Visualization of two soft manipulator policies. Top: end-effector tracking of a moving target traveling at 0.5 m/s. Middle: static target reaching through eight random 3D obstacles. Note how the policy first attempts to take the shortest path to the target sphere and upon encountering resistance, starts to probe for gaps within the obstacles, demonstrating emergent tactile behavior. Bottom: discounted return (DR) versus wall-clock time for the 3D contact case using Elastica and DisMech as the simulator. DisMech attains over $17\times$ faster training per iteration.
• Figure 2: Discrete rod schematic. A continuous centerline can be seen discretized into nodes $\mathbf q_i$. Each discrete edge is represented by a reference frame $\{\mathbf d^i_1, \mathbf d^i_2, \mathbf t^i \}$ and a material frame $\{\mathbf m^i_1, \mathbf m^i_2, \mathbf t^i \}$, which are used to compute bending and twisting deformations at interior nodes.
• Figure 3: Comparison of the discounted return with respect to both environment steps (top row) and training wall-clock time (bottom row) across five random seeds. For each task, we can see almost identical convergence rates regardless of the choice of simulator with respect to environment steps. Minimal variance can be observed aside from 2D Tight Obstacles, given the narrow success condition. Despite the similar convergence properties, plotting the same data against wall-clock time shows the immense speed benefits of leveraging implicit time-stepping.