Dexterous Contact-Rich Manipulation via the Contact Trust Region

TL;DR

This work reframes contact-rich manipulation by introducing the Contact Trust Region (CTR), a convex, locally valid description that respects the unilateral nature of contact, and contrasts it with traditional ellipsoidal trust regions. It builds a fast, contact-implicit MPC via CTR/R-CTR, enabling local dexterous control, and couples it with a roadmap-based global planner to stitch local plans into long-horizon manipulation trajectories. The method leverages the Convex Quasidynamic Differentiable Contact (CQDC) model to obtain smoothed gradients and dual information, facilitating stable planning and control even in high-contact scenarios. Hardware and high-fidelity simulations on IiwaBimanual and AllegroHand demonstrate substantial reductions in computation time and improved robustness compared with RL-based approaches, with offline roadmap construction taking minutes and online inference on a laptop, highlighting practical impact for real-time, contact-rich manipulation.

Abstract

What is a good local description of contact dynamics for contact-rich manipulation, and where can we trust this local description? While many approaches often rely on the Taylor approximation of dynamics with an ellipsoidal trust region, we argue that such approaches are fundamentally inconsistent with the unilateral nature of contact. As a remedy, we present the Contact Trust Region (CTR), which captures the unilateral nature of contact while remaining efficient for computation. With CTR, we first develop a Model-Predictive Control (MPC) algorithm capable of synthesizing local contact-rich plans. Then, we extend this capability to plan globally by stitching together local MPC plans, enabling efficient and dexterous contact-rich manipulation. To verify the performance of our method, we perform comprehensive evaluations, both in high-fidelity simulation and on hardware, on two contact-rich systems: a planar IiwaBimanual system and a 3D AllegroHand system. On both systems, our method offers a significantly lower-compute alternative to existing RL-based approaches to contact-rich manipulation. In particular, our Allegro in-hand manipulation policy, in the form of a roadmap, takes fewer than 10 minutes to build offline on a standard laptop using just its CPU, with online inference taking just a few seconds. Experiment data, video and code are available at ctr.theaiinstitute.com.

Figures (25)

• Figure 1: Hardware experiments illustrating the utility of our proposed method in contact-rich manipulation. Left: Dexterous in-hand manipulation with the Allegro hand moving a cube. Right: Whole-body manipulation with bimanual iiwas moving a bucket.
• Figure 2: A-CTR illustrations for the system in \ref{['fig:1d-pushing-schematic']}b at $(\bar{q}^\mathrm{o}, \bar{q}^\mathrm{a}) = (0.2, 0)$. Sub-figures show how (a) the signed distance function $\phi$, (b) the contact force $\lambda$, (c) the object configuration ${q^\mathrm{o}_{\texttt{+}}}$ and (d) the robot configuration ${q^\mathrm{a}_{\texttt{+}}}$ change as a function of the action $u$. In every sub-figure, the black solid line represents the true, non-smooth dynamics \ref{['eq:q_dynamic_f']}. The markers (squares and circles) represent linearization points. In (a) and (b), the deeper-colored dotted lines represent the linearization at $\bar{u}_0 = 0$, and the lighter-colored lines represent the linearization at $\bar{u}_1=0.02$. The shaded regions represent the feasible set of the corresponding color. For instance, the dark blue shaded region in (a) represents $\hat{\phi}\left(\bar{q}=(0.2, 0), \bar{u}=0\right) \geq 0$; the light green region in (b) represents $\hat{\lambda}_+\left(\bar{q}=(0.2, 0), \bar{u}=0.02\right) \geq 0$. In (c) and (d), the dotted lines show parts of the linearizations that satisfy only the dual constraints; the thick shaded lines around dotted lines satisfy both the primal and dual constraints.
• Figure 3: (a) Schematic of the 1-dimensional pusher system used in \ref{['ex:pushing-actr']}. (b) corresponds to the configuration $({q^\mathrm{o}}, {q^\mathrm{a}}) = (0.2, 0)$.
• Figure 4: (a) Nominal configuration and actions for \ref{['ex:squeezing-actr']}. $u_0$ and $u_1$ are the position commands of the left and right ball, respectively. (b) Samples in the action space that satisfy the primal \ref{['eq:full_contact_trust_region:primal_feasibility']} and dual \ref{['eq:full_contact_trust_region:dual_feasibility']} feasibility constraints for different nominal actions.
• Figure 5: Visualization of the A-CTR $\mathcal{S}^\mathcal{A}_{\mathbf{\Sigma},\kappa}$ (primal and dual), RA-CTR $\tilde{\mathcal{S}}^\mathcal{A}_{\mathbf{\Sigma},\kappa}$ (dual only) and the action-only object motion set $\mathcal{M}^{\mathcal{A},\mathrm{o}}_{\mathrm{\Sigma}, \kappa}$ under the different linearization points shown in the illustrations. In the "Trust Region" row, the "True Samples" subplot is obtained by uniformly sampling 2000 points from the ball $\left\lVert{\delta u}\right\rVert \leq 0.1$; samples in the following columns are obtained by rejecting samples that do not satisfy the respective constraints. In the "Object Motion Set" row, the "True Samples" subplot is obtained by passing the $\delta u$ samples from the trust region subplot above through the true contact dynamics \ref{['eq:q_dynamic_f']}; samples in the following columns are obtained by mapping the $\delta u$ samples from the corresponding trust region subplot through the linear map defined in \ref{['eq:action-only-motion_set']}.

Theorems & Definitions (14)

• Lemma 1: Taylor Approximation
• Definition 1: Contact Trust Region
• Definition 2: Action-only Contact Trust Region
• Example 1: A-CTR for 1D Pushing
• Example 2: A-CTR for 1D Squeezing
• Definition 3: Relaxed Contact Trust Region
• Definition 4: Relaxed Action-only Contact Trust Region
• Definition 5: Motion Set
• Definition 6: Action-only Motion Set
• Example 3: A-CTRs and Motion Sets