Contact-Implicit Model Predictive Control for Dexterous In-hand Manipulation: A Long-Horizon and Robust Approach

TL;DR

A high-level contact-implicit model predictive controller is designed to generate real-time contact plans executed by the low-level tracking controller, which allows the robot to robustly perform long-horizon in-hand manipulation without predefined contact sequences or separate planning procedures.

Abstract

Dexterous in-hand manipulation is an essential skill of production and life. However, the highly stiff and mutable nature of contacts limits real-time contact detection and inference, degrading the performance of model-based methods. Inspired by recent advances in contact-rich locomotion and manipulation, this paper proposes a novel model-based approach to control dexterous in-hand manipulation and overcome the current limitations. The proposed approach has an attractive feature, which allows the robot to robustly perform long-horizon in-hand manipulation without predefined contact sequences or separate planning procedures. Specifically, we design a high-level contact-implicit model predictive controller to generate real-time contact plans executed by the low-level tracking controller. Compared to other model-based methods, such a long-horizon feature enables replanning and robust execution of contact-rich motions to achieve large displacements in-hand manipulation more efficiently; Compared to existing learning-based methods, the proposed approach achieves dexterity and also generalizes to different objects without any pre-training. Detailed simulations and ablation studies demonstrate the efficiency and effectiveness of our method. It runs at 20Hz on the 23-degree-of-freedom, long-horizon, in-hand object rotation task.

Figures (8)

• Figure 1: Overview of the proposed method for long-horizon in-hand manipulation. The reference generator infers how to make and break contacts in the system's neighborhood based on desired object motions. Then, the contact controller addresses modeling errors with tactile feedback and executes the inferred contacts to the fullest. Finally, joint-level impedance control is used to execute the contact-rich motions. The proposed method can seamlessly generalize among different objects.
• Figure 2: Block diagram of the proposed method. The long-horizon contact-rich manipulation is accomplished by the contact reference generator and the contact controller working as a hierarchical structure.
• Figure 3: Modeling of the contact controller. The frames $\bm{O}_r,\bm{O}_c,\bm{O}_d$, frame displacements $d\bm{p}_{c,r},d\bm{p}_{r,d}$, contact jacobians $\bm{G}_i,\bm{J}_i$, stiffness matrices $\bm{K}_c,\bm{K}_r,\bm{K}_o$ are illustrated. Note that $\bm{O}_d$ can be obtained with forward kinematics $\bm{FK}(\bm{\xi}_d)$. The model compliance creates a coupling effect between force and motion.
• Figure 4: Two in-hand manipulation systems used in the experiments. The object being manipulated is painted in blue. (a) The orange cylinder represents a damped hinge. (b) The brown body represents the supporting plane.
• Figure 5: The relationship between $\kappa$ and optimization results. The cost of DDP, the object's yaw angle, and the control input norms are plotted for 50 steps. Warmer colors indicate a stronger smoothing effect.