DittoGym: Learning to Control Soft Shape-Shifting Robots

TL;DR

DittoGym formalizes control of reconfigurable soft robots as a high-dimensional reinforcement learning problem using a continuous 2D muscle-field action space, integrated with an elasto-plastic Material Point Method simulation. It introduces Coarse-to-Fine Policy (CFP), a fully-convolutional architecture that first explores with a coarse action grid and then refines actions with a high-resolution residual policy guided by a gating mask, enabling fine-grained morphology changes during tasks. The authors present DittoGym, a benchmark suite of eight long-horizon tasks that require dynamic morphology changes, and demonstrate that CFP outperforms baselines in sample efficiency and reliability, enabling robots to morph multiple times within a sequence. This framework advances learning-based control for lifelike shape-shifting robots and provides a scalable platform for evaluating morphology-aware policies in soft robotics.

Abstract

Robot co-design, where the morphology of a robot is optimized jointly with a learned policy to solve a specific task, is an emerging area of research. It holds particular promise for soft robots, which are amenable to novel manufacturing techniques that can realize learned morphologies and actuators. Inspired by nature and recent novel robot designs, we propose to go a step further and explore the novel reconfigurable robots, defined as robots that can change their morphology within their lifetime. We formalize control of reconfigurable soft robots as a high-dimensional reinforcement learning (RL) problem. We unify morphology change, locomotion, and environment interaction in the same action space, and introduce an appropriate, coarse-to-fine curriculum that enables us to discover policies that accomplish fine-grained control of the resulting robots. We also introduce DittoGym, a comprehensive RL benchmark for reconfigurable soft robots that require fine-grained morphology changes to accomplish the tasks. Finally, we evaluate our proposed coarse-to-fine algorithm on DittoGym and demonstrate robots that learn to change their morphology several times within a sequence, uniquely enabled by our RL algorithm. More results are available at https://suninghuang19.github.io/dittogym_page/.

Figures (12)

• Figure 1: Reconfigurable soft robots. In this paper, we address challenges in controlling reconfigurable - or shape-shifting - robots, who can change their morphology to accomplish desired tasks. The figure above illustrates a task where a circular robot needs to alter its body shape to fit within a confined chamber to manipulate the square cargo to the target point (right). We introduce a benchmark with 8 tasks for shape-shifting robots, which we dub "DittoGym" inspired by a shape-shifting Pokémon (left).
• Figure 2: (a) The soft reconfigurable robot, initialized with a specific shape, is ready to venture into the task. (b) Instead of directly controlling material points in the robot, the policy first applies actions to the action grid, which corresponds to the grid in the material point method (MPM). (c) Each particle aggregates actuation signals from its adjacent grid points during the grid-to-particle distribution stage. (d) Under the principles of Cauchy stress and the von Mises yield criterion, we can actuate the particles in a physically plausible manner, thereby changing the robot's morphology and state.
• Figure 3: Illustration of CFP. In stage one, we train a coarse policy to efficiently explore the action space and discover meaningful action patterns. Subsequently, in stage two, we employ a coarse-to-fine approach to train a high-resolution residual policy that delves deeper into optimizing the actions for improved performance.
• Figure 4: Illustration of reconfigurable robots in diverse tasks. This figure showcases selected visualization results from DittoGym. Notably, policies trained under the guidance of CFP exhibit precise control over highly reconfigurable robots, enabling them to successfully accomplish their respective tasks. More visualization results can be found in Appendix \ref{['app:exp_demo']}.
• Figure 5: Performance of expert policies under different action resolutions. Expert policies using the highest resolution (fine) can achieve much higher episode rewards compared to those with medium or coarse resolutions, illustrating the tasks in DittoGym require fine-grained morphology changes.