SOFTMAP: Sim2Real Soft Robot Forward Modeling via Topological Mesh Alignment and Physics Prior

Abstract

While soft robot manipulators offer compelling advantages over rigid counterparts, including inherent compliance, safe human-robot interaction, and the ability to conform to complex geometries, accurate forward modeling from low-dimensional actuation commands remains an open challenge due to nonlinear material phenomena such as hysteresis and manufacturing variability. We present SOFTMAP, a sim-to-real learning framework for real-time 3D forward modeling of tendon-actuated soft finger manipulators. SOFTMAP combines four components: (1) As-Rigid-As-Possible (ARAP)-based topological alignment that projects simulated and real point clouds into a shared, topologically consistent vertex space; (2) a lightweight MLP forward model pretrained on simulation data to map servo commands to full 3D finger geometry; (3) a residual correction network trained on a small set of real observations to predict per-vertex displacement fields that compensate for sim-to-real discrepancies; and (4) a closed-form linear actuation calibration layer enabling real-time inference at 30 FPS. We evaluate SOFTMAP on both simulated and physical hardware, achieving state-of-the-art shape prediction accuracy with a Chamfer distance of 0.389 mm in simulation and 3.786 mm on hardware, millimeter-level fingertip trajectory tracking across multiple target paths, and a 36.5% improvement in teleoperation task success over the baseline. Our results show that SOFTMAP provides a data-efficient approach for 3D forward modeling and control of soft manipulators.

Figures (8)

• Figure 1: SOFTMAP overview. Simulation and real multi-view data are aligned via ARAP into a shared topology, enabling a lightweight learned forward model to predict full 3D soft finger geometry from servo commands for real-time control and teleoperation.
• Figure 2: SOFTMAP pipeline. Simulation data and real multi-view images are independently processed into 3D representations, then encoded via ARAP into a shared topologically consistent vertex space to form an aligned dataset. A learned MLP maps servo commands to 3D shape predictions, which are subsequently refined by a residual correction network. The resulting forward model enables downstream applications, including trajectory generation and real-time teleoperation.
• Figure 3: Simulation environment. The soft finger is modeled in the SOFA Framework using a Neo-Hookean hyperelastic material with four embedded tendons. Actuation commands $u_{\text{sim}}^{(i)} \in \mathbb{R}^2$ are swept across a dense grid to generate shape observations $Y_{\text{sim}}^{(i)} \in \mathbb{R}^{548 \times 3}$, forming the simulation dataset used to pretrain SOFTMAP's model.
• Figure 4: Real-world data collection setup. Two RGB cameras are mounted at distinct viewpoints around a soft finger actuated by an xArm 7 robot. For each servo command, synchronized multi-view images are captured and fed into our reconstruction pipeline to produce 3D point clouds used for real world deployment.
• Figure 5: Evaluation Comparison. Left: Point Cloud Comparisons in Simulation Data; Right: Point Cloud Comparisons in Real Data