Adaptive Contact-Implicit Model Predictive Control with Online Residual Learning

TL;DR

This work tackles the difficulty of controlling multi-contact robotic systems with uncertain contact events by introducing a real-time adaptive MPC framework that online learns a residual on top of a physics-based hybrid model. The residual focuses on correcting contact dynamics via the complementarity constraints, enabling on-the-fly adaptation at real-time rates (about 20 Hz) and improving performance in contact-rich manipulation tasks. The method is validated through synthetic examples and hardware experiments, including rolling unknown objects and handling non-smooth surfaces, demonstrating that adaptation enables success where pure model-based controllers fail. Overall, the approach enhances robustness and versatility for dexterous manipulation in uncertain environments by integrating online residual learning with fast, contact-implicit MPC.

Abstract

The hybrid nature of multi-contact robotic systems, due to making and breaking contact with the environment, creates significant challenges for high-quality control. Existing model-based methods typically rely on either good prior knowledge of the multi-contact model or require significant offline model tuning effort, thus resulting in low adaptability and robustness. In this paper, we propose a real-time adaptive multi-contact model predictive control framework, which enables online adaption of the hybrid multi-contact model and continuous improvement of the control performance for contact-rich tasks. This framework includes an adaption module, which continuously learns a residual of the hybrid model to minimize the gap between the prior model and reality, and a real-time multi-contact MPC controller. We demonstrated the effectiveness of the framework in synthetic examples, and applied it on hardware to solve contact-rich manipulation tasks, where a robot uses its end-effector to roll different unknown objects on a table to track given paths. The hardware experiments show that with a rough prior model, the multi-contact MPC controller adapts itself on-the-fly with an adaption rate around 20 Hz and successfully manipulates previously unknown objects with non-smooth surface geometries.

Figures (6)

• Figure 1: Given an initial guess that the object is a rigid sphere, the controller adapts its model of the governing contact dynamics to roll and push real fruits (orange, lime) with a Franka Emika Panda arm, tracking a desired motion.
• Figure 2: Key elements of the adaptive MPC framework. Given proprioception and visual data, our method learns a residual multi-contact model at $20$ Hz, which we use for real-time control.
• Figure 3: A "contact event" refers to a situation where actual physical contact is occurring, while "contact prediction" pertains to instances where the model anticipates contact, potentially inaccurately. Our method can produce meaningful gradients, even when there is no actual contact event (yellow region). The only scenario in which a zero gradient is produced is when the model and data both agree that there is no contact (white region).
• Figure 4: Stabilization of the cart-pole system and convergence of residual.
• Figure 5: Rolling a rigid ball and fruits (left: ball, middle: orange, right: lime) starting with an inaccurate model.

Theorems & Definitions (2)

• Definition 1
• Definition 2