ContactSDF: Signed Distance Functions as Multi-Contact Models for Dexterous Manipulation

TL;DR

ContactSDF introduces a differentiable, SDF-based framework to model multi-contact dexterous manipulation by pairing a Collision-Detection SDF (C-SDF) with a Time-Stepping SDF (D-SDF). This dual-SDF approach yields explicit, differentiable forward dynamics suitable for MPC and learning, removing the need for non-smooth complementarity-based optimizations. The authors develop ContactSDF-MPC and demonstrate data-efficient learning of D-SDF parameters from on-MPC data, achieving real-time control on simulated tasks and hardware (Allegro hand) with rapid convergence. The work offers a practical path to efficient, model-based control of contact-rich manipulation, with potential for rapid hardware adaptation and scalable integration into learning pipelines.

Abstract

In this paper, we propose ContactSDF, a method that uses signed distance functions (SDFs) to approximate multi-contact models, including both collision detection and time-stepping routines. ContactSDF first establishes an SDF using the supporting plane representation of an object for collision detection, and then uses the generated contact dual cones to build a second SDF for time-stepping prediction of the next state. Those two SDFs create a differentiable and closed-form multi-contact dynamic model for state prediction, enabling efficient model learning and optimization for contact-rich manipulation. We perform extensive simulation experiments to show the effectiveness of ContactSDF for model learning and real-time control of dexterous manipulation. We further evaluate the ContactSDF on a hardware Allegro hand for on-palm reorientation tasks. Results show with around 2 minutes of learning on hardware, the ContactSDF achieves high-quality dexterous manipulation at a frequency of 30-60Hz. Project page https://yangwen-1102.github.io/contactsdf.github.io/

Figures (10)

• Figure 1: 2D contact detection example. (a) Collision detection in a manipulation system can be viewed as a distance query to a convex object, where query points are taken from the robot and environment. (b) The truth distance field $\phi_{\mathcal{G}}(\boldsymbol{x}_{\text{query}})$ in (\ref{['equ.phi']}) for the object. (c) Distance field approximated by (\ref{['equ.closest_dist']}). (d) The $\mathop{\mathrm{\texttt{C-SDF}}}\nolimits_{\mathcal{G}}$ distance field (\ref{['equ.lse_dist']}) with large $\sigma$. (e) The $\mathop{\mathrm{\texttt{C-SDF}}}\nolimits_{\mathcal{G}}$ distance field (\ref{['equ.lse_dist']}) with small $\sigma$.
• Figure 2: Pipeline of learning $\mathop{\mathrm{\texttt{D-SDF}}}\nolimits$ from on-MPC data taskdrivenjin.
• Figure 3: Learning $\mathop{\mathrm{\texttt{D-SDF}}}\nolimits$ with on-MPC data in the three-ball manipulation tasks. Left and right panel shows the normalized accumulated cost of environment rollout, evaluated by $c_T$ (\ref{['equ:mpc_detail_cost_fn']}), and normalized model prediction loss (\ref{['equ.model_learning_loss']}), respectively. Each plot is based on five learning trials.
• Figure 4: Evaluation examples for ContactSDF-MPC in the three-ball manipulation. The first and second rows show the initial and final scenes of a task, respectively.
• Figure 5: Learning $\mathop{\mathrm{\texttt{D-SDF}}}\nolimits$ with on-MPC data in the Allegro hand on-palm reorientation tasks. The blue and red lines show the normalized accumulated cost of environment rollout, evaluated by $c_T$ (\ref{['equ:mpc_detail_cost_fn']}), and normalized model prediction loss (\ref{['equ.model_learning_loss']}) along with the learning steps. Each plot is based on five learning trials, showing the mean and standard deviation.