Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Complementarity-Free Multi-Contact Modeling and Optimization for Dexterous Manipulation

Wanxin Jin

TL;DR

The paper tackles the barrier of real-time dexterous manipulation posed by non-smooth multi-contact dynamics arising from complementarity constraints. It introduces a complementarity-free multi-contact model derived via the dual of optimization-based contact formulations, yielding closed-form time stepping, differentiability, and automatic adherence to the Coulomb friction cone with fewer hyperparameters. Integrated into a contact-implicit MPC framework, the approach achieves state-of-the-art performance across fingertip in-air manipulation, TriFinger in-hand manipulation, and Allegro hand on-palm reorientation, with an average success rate of 96.5%, average reorientation error of 11°, position error of 7.8 mm, and real-time MPC in the 50–100 Hz range. The method demonstrates substantial practical impact by enabling fast, accurate, and generalizable dexterous manipulation across varied objects and robot morphologies, while also outlining avenues for extending to dynamic settings and global contact reasoning.

Abstract

A significant barrier preventing model-based methods from achieving real-time and versatile dexterous robotic manipulation is the inherent complexity of multi-contact dynamics. Traditionally formulated as complementarity models, multi-contact dynamics introduces non-smoothness and combinatorial complexity, complicating contact-rich planning and optimization. In this paper, we circumvent these challenges by introducing a lightweight yet capable multi-contact model. Our new model, derived from the duality of optimization-based contact models, dispenses with the complementarity constructs entirely, providing computational advantages such as closed-form time stepping, differentiability, automatic satisfaction with Coulomb friction law, and minimal hyperparameter tuning. We demonstrate the effectiveness and efficiency of the model for planning and control in a range of challenging dexterous manipulation tasks, including fingertip 3D in-air manipulation, TriFinger in-hand manipulation, and Allegro hand on-palm reorientation, all performed with diverse objects. Our method consistently achieves state-of-the-art results: (I) a 96.5% average success rate across all objects and tasks, (II) high manipulation accuracy with an average reorientation error of 11° and position error of 7.8mm, and (III) contact-implicit model predictive control running at 50-100 Hz for all objects and tasks. These results are achieved with minimal hyperparameter tuning.

Complementarity-Free Multi-Contact Modeling and Optimization for Dexterous Manipulation

TL;DR

The paper tackles the barrier of real-time dexterous manipulation posed by non-smooth multi-contact dynamics arising from complementarity constraints. It introduces a complementarity-free multi-contact model derived via the dual of optimization-based contact formulations, yielding closed-form time stepping, differentiability, and automatic adherence to the Coulomb friction cone with fewer hyperparameters. Integrated into a contact-implicit MPC framework, the approach achieves state-of-the-art performance across fingertip in-air manipulation, TriFinger in-hand manipulation, and Allegro hand on-palm reorientation, with an average success rate of 96.5%, average reorientation error of 11°, position error of 7.8 mm, and real-time MPC in the 50–100 Hz range. The method demonstrates substantial practical impact by enabling fast, accurate, and generalizable dexterous manipulation across varied objects and robot morphologies, while also outlining avenues for extending to dynamic settings and global contact reasoning.

Abstract

A significant barrier preventing model-based methods from achieving real-time and versatile dexterous robotic manipulation is the inherent complexity of multi-contact dynamics. Traditionally formulated as complementarity models, multi-contact dynamics introduces non-smoothness and combinatorial complexity, complicating contact-rich planning and optimization. In this paper, we circumvent these challenges by introducing a lightweight yet capable multi-contact model. Our new model, derived from the duality of optimization-based contact models, dispenses with the complementarity constructs entirely, providing computational advantages such as closed-form time stepping, differentiability, automatic satisfaction with Coulomb friction law, and minimal hyperparameter tuning. We demonstrate the effectiveness and efficiency of the model for planning and control in a range of challenging dexterous manipulation tasks, including fingertip 3D in-air manipulation, TriFinger in-hand manipulation, and Allegro hand on-palm reorientation, all performed with diverse objects. Our method consistently achieves state-of-the-art results: (I) a 96.5% average success rate across all objects and tasks, (II) high manipulation accuracy with an average reorientation error of 11° and position error of 7.8mm, and (III) contact-implicit model predictive control running at 50-100 Hz for all objects and tasks. These results are achieved with minimal hyperparameter tuning.
Paper Structure (50 sections, 2 theorems, 57 equations, 17 figures, 8 tables)

This paper contains 50 sections, 2 theorems, 57 equations, 17 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Rigid Body Multi-contact Models
  4. Nonconvex Complementarity Contact Models
  5. Cone Complementarity Contact Models
  6. Optimization-based Contact Models
  7. Differentiable Contact Models
  8. Why is our model new?
  9. Planning and Control with Contact Dynamics
  10. Reinforcement Learning for Dexterous Manipulation
  11. Preliminary and Problem Statement
  12. Optimization-based Quasi-Dynamic Contact Model
  13. Model Predictive Control
  14. Problem: Contact-implicit MPC for Dexterous Manipulation
  15. Complementarity-free Multi-contact Model
  16. ...and 35 more sections

Key Result

Lemma 1

The dual solution to the regularized dual problem (equ.dual_prob_relax) satisfies the following dual complementarity constraints:

Figures (17)

  • Figure 1: The physical interpretation of the proposed complementarity-free multi-contact model (\ref{['equ.key_primal']}) using a 2D fingertip manipulation example. (a) The dual cone constraints, depicted by red shaded areas, between the object and the ground and fingertips. (b) The displacement (green) caused solely by the non-contact force $\boldsymbol{b}$ applied to the system to move the object from dashed position to solid position. (c) The spring-like contact force (blue) resulting from the penetration (black) of the contact dual cone. (d) An extension: a damping effect (purple) is added to the contact force term.
  • Figure 2: Complementarity-free contact-implicit MPC
  • Figure 3: (a) The predicted scene of pushing 10 boxes at different steps, by the proposed model (upper row) and QP-based model (bottom row). (b) Timing comparison for one-step prediction. The tests were conducted on a machine with an Apple M2 Pro chip.
  • Figure 4: Left: a sphere is accelerated between two frictional planes under an external force (green arrow). Right: the tangential velocity of the sphere versus time. NCP horak2019similaritiesstewardtrinklePolyhedron and BLCP erleben2007velocity contact models capture the acceleration, while the velocity plateaus for the CCP anitescu2010iterative and Convex anitescu2006optimization models due to the "sheath effect" induced by the dual cone constraints. The proposed complementarity-free model mitigates such issue because it allows for "violation" of the dual cone constraints while also introducing significant smoothness.
  • Figure 5: Left: an unactuated box sliding with initial horizontal velocity. Middle and right: the horizontal velocity and vertical position trajectories, respectively. QP-based model (\ref{['equ.primal_prob']}) exhibits vertical motion artifacts due to the enforcement of dual cone constraints, the proposed model significantly mitigates these effects by relaxing the dual cone constraints. Compared to MuJoCo, our model yields noticeably smoother velocity profiles.
  • ...and 12 more figures

Theorems & Definitions (2)

  • Lemma 1
  • Lemma 2