Learning Soft Robotic Dynamics with Active Exploration

TL;DR

The paper tackles the challenge of learning generalizable soft-robot dynamics by introducing SoftAE, an uncertainty-aware active exploration framework. SoftAE uses probabilistic ensembles to estimate epistemic uncertainty and guides data collection toward underexplored state–action regions, enabling task-agnostic dynamics models that generalize to unseen objectives and morphologies. Through extensive experiments on simulated platforms (soft continuum arm, deformable fish in fluid, and musculoskeletal leg) and a real pneumatically actuated SoPrA arm, SoftAE achieves more accurate dynamics models, better zero-shot planning, and robustness to noise and nonlinear material effects compared with random exploration and task-specific model-based RL. The results demonstrate the practicality of uncertainty-driven exploration for scalable, reusable dynamics models in soft robotics, advancing autonomous and data-efficient control for compliant morphologies.

Abstract

Soft robots offer unmatched adaptability and safety in unstructured environments, yet their compliant, high-dimensional, and nonlinear dynamics make modeling for control notoriously difficult. Existing data-driven approaches often fail to generalize, constrained by narrowly focused task demonstrations or inefficient random exploration. We introduce SoftAE, an uncertainty-aware active exploration framework that autonomously learns task-agnostic and generalizable dynamics models of soft robotic systems. SoftAE employs probabilistic ensemble models to estimate epistemic uncertainty and actively guides exploration toward underrepresented regions of the state-action space, achieving efficient coverage of diverse behaviors without task-specific supervision. We evaluate SoftAE on three simulated soft robotic platforms -- a continuum arm, an articulated fish in fluid, and a musculoskeletal leg with hybrid actuation -- and on a pneumatically actuated continuum soft arm in the real world. Compared with random exploration and task-specific model-based reinforcement learning, SoftAE produces more accurate dynamics models, enables superior zero-shot control on unseen tasks, and maintains robustness under sensing noise, actuation delays, and nonlinear material effects. These results demonstrate that uncertainty-driven active exploration can yield scalable, reusable dynamics models across diverse soft robotic morphologies, representing a step toward more autonomous, adaptable, and data-efficient control in compliant robots.

Figures (10)

• Figure 1: Learning soft robotic dynamics with active exploration. (A--D) Soft robotic platforms studied in this work: (A) soft continuum arm, (B) deformable fish in fluid, (C) musculoskeletal (MSK) leg with electrohydraulic actuation (yellow arrows), and (D) real-world pneumatically actuated arm (SoPrA). Their nonlinear, high-dimensional, and deformable dynamics pose significant challenges for generalizable model learning. (E) We address this with SoftAE, an active exploration strategy that autonomously collects informative and diverse data for training dynamics models. During dynamics learning (top), Random exploration is unguided, and task-oriented method overfits. SoftAE instead targets regions of high model uncertainty, improving data coverage and model accuracy. This enables robust generalization to diverse downstream tasks (bottom), as (F) demonstrated across all platforms.
• Figure 2: Active exploration enables robust zero-shot task performance across diverse soft robotic systems. We evaluate SoftAE against baseline exploration strategies across three soft robotic platforms: (A) a soft continuum arm, (B) a deformable fish swimming in fluid, and (C) a musculoskeletal (MSK) leg actuated by electrohydraulic muscles and a DC motor. H-UCRL, a task-specific model-based baseline, is trained only on tasks (i), (iii), (v), and (vii), while the remaining tasks are held out as unseen. Across tasks, SoftAE consistently produces task-aligned and physically plausible behaviors, while Random and H-UCRL often fail to generalize to the hard or unseen scenarios. Returns are averaged over 10 random seeds for tasks (i) and (ii), and 5 seeds for the remaining tasks.
• Figure 3: Active exploration improves data coverage, model accuracy, and scales to high-dimensional dynamics.(A) Spatial coverage of collected data for the soft continuum arm, visualized by projecting tip positions onto the $x$--$z$ plane. Heatmaps show visitation frequency (counts) per spatial bin. SoftAE explores more broadly and uniformly across the reachable workspace compared to Random and H-UCRL, enabling the collection of diverse and informative training data for dynamics learning. (B) Normalized mean squared error (MSE) of learned dynamics models on a held-out set of 17,451 transitions. SoftAE achieves the lowest error, indicating more globally accurate models. (C) Return curves for downstream task performance on a soft arm with a 136-dimensional state space, reaching (i) close and (ii) far targets. SoftAE maintains high performance even in this high-dimensional setting, matching H-UCRL on its trained task and outperforming baselines on the more challenging unseen task. Returns are averaged over 10 seeds.
• Figure 4: Real-world dynamics learning and task performance on a pneumatically actuated soft arm. (A) SoPrA: a fiber-reinforced, pneumatically actuated continuum arm with motion capture via marker rings. (B) Reaching tasks for (ix) close and (x) far targets. Top: front view; bottom: bottom view. Models are trained using Random, H-UCRL (task-specific, trained only on (ix)), or SoftAE (ours). Yellow dots indicate target positions. (C) Return curves show all methods perform similarly on task (ix), while SoftAE outperforms on the harder, unseen task (x). Returns are averaged over 3 random seeds, shaded regions denote standard deviation. (D) Quantitative evaluation over the same set of 20 random targets shows that SoftAE achieves the highest mean return and lowest tip position error compared to baselines. (E) Trajectory tracking performance on two reference trajectories: square (top row) and lemniscate (bottom row). SoftAE tracks both shapes more accurately than baselines.
• Figure S1: Comparison of different uncertainty-driven active exploration strategies. Same as return plots in Figure \ref{['fig:2_sim_tasks']} but with two additional active exploration baselines: Mean-AE and PETS-AE. Mean-AE plans trajectories using the mean model $\bm{\mu}_n$, PETS-AE uses trajectory sampling from the model posterior to improve robustness to model inaccuracies.