EHC-MM: Embodied Holistic Control for Mobile Manipulation

TL;DR

This work tackles unified control for mobile manipulation by addressing the inefficiency of decoupled base and arm planning. It introduces Embodied Holistic Control for Mobile Manipulation (EHC-MM), which uses an embodied function $\text{sig}(\omega)$ to dynamically balance base motion and arm manipulation within a Quadratic Programming (QP) framework, while accounting for obstacle avoidance and joint limits. A Monitoring-Position-Based Servoing (MPBS) module complements EHC by maintaining target visibility during motion, and GS-Net* enhances stable grasp pose generation. Extensive simulations and real-world experiments show higher success rates and greater time efficiency, with real deployments achieving up to 95.6% grasp success.

Abstract

Mobile manipulation typically entails the base for mobility, the arm for accurate manipulation, and the camera for perception. The principle of Distant Mobility, Close Grasping(DMCG) is essential for holistic control. We propose Embodied Holistic Control for Mobile Manipulation(EHC-MM) with the embodied function of sig(w): By formulating the DMCG principle as a Quadratic Programming (QP) problem, sig(w) dynamically balances the robot's emphasis between movement and manipulation with the consideration of the robot's state and environment. In addition, we propose the Monitor-Position-Based Servoing (MPBS) with sig(w), enabling the tracking of the target during the operation. This approach enables coordinated control among the robot's base, arm, and camera, enhancing task efficiency. Through extensive simulations and real-world experiments, our approach significantly improves both the success rate and efficiency of mobile manipulation tasks, achieving a 95.6% success rate in real-world scenarios and a 52.8% increase in time efficiency.

Figures (6)

• Figure 1: EHC-MM. The function $\textbf{sig}(\omega)$ of EHC-MM dynamically balances the robot’s emphasis between movement and manipulation with the consideration of the robot's state and environment. It coordinates all joints of the robot simultaneously and has been validated both in simulation and real-world scenarios.
• Figure 2: Framework of EHC-MM. The core of the framework is the embodied function $\textbf{sig}(\omega)$, which dynamically balances the robot’s focus between movement and manipulation, taking into account both the robot’s state and its environment. Combined with MPBS, $\textbf{sig}(\omega)$ improves the robot’s ability to track targets. Moreover, our proposed GS-Net* enhances the stability of grasp poses. Overall, this framework significantly improves the efficiency of mobile manipulation.
• Figure 3: A comparison of the mobile manipulation between PBS and MPBS.
• Figure 4: Top-10 and top-1 grasp poses for GS-Net and our GS-Net*
• Figure 5: Setup of experiment-2. The Fixed Grasping Position in the figure is only set for baseline TSMM.