Interactive Navigation with Adaptive Non-prehensile Mobile Manipulation

TL;DR

This letter introduces a framework for interactive navigation through adaptive non-prehensile mobile manipulation through a learned $SE(2)$ dynamics representation integrated into a Model Predictive Path Integral (MPPI) controller to guide the robot’s motion.

Abstract

This paper introduces a framework for interactive navigation through adaptive non-prehensile mobile manipulation. A key challenge in this process is handling objects with unknown dynamics, which are difficult to infer from visual observation. To address this, we propose an adaptive dynamics model for common movable indoor objects via learned SE(2) dynamics representations. This model is integrated into Model Predictive Path Integral (MPPI) control to guide the robot's interactions. Additionally, the learned dynamics help inform decision-making when navigating around objects that cannot be manipulated.Our approach is validated in both simulation and real-world scenarios, demonstrating its ability to accurately represent object dynamics and effectively manipulate various objects. We further highlight its success in the Navigation Among Movable Objects (NAMO) task by deploying the proposed framework on a dynamically balancing mobile robot, Shmoobot. Project website: https://cmushmoobot.github.io/AdaptivePushing/.

Figures (8)

• Figure 1: Interactive Navigation with Adaptive Pushing: Time-lapse of interactive navigation with adaptive pushing on Shmoobot, demonstrating the robot's ability to either push or plan a collision-free path (CF-Path) to navigate around an obstructing object based on its manipulability. The figure showcases two common object types: a light, manipulable box and a non-manipulable wheeled cart (e.g., a wheelchair with locked wheels).
• Figure 2: Overall framework: In the offline phase, we first create simulation environments in Bullet coumans2016pybullet with three types of objects with randomized physical properties and train the basis function $\psi$ and robot model $R$. The learned $\psi$ are then used as parts of the adaptive dynamics, as described in Section.\ref{['sec:AdaptivePushing']}. In step 2, we utilize the adaptive dynamics to formulate the controller for interactive navigation. Given a pre-built map, the controller switches between adaptive pushing and collision-free navigation based on whether the object is manipulable or is blocking the way. The details of the interactive navigation framework are described in Sec. \ref{['sec:InteractiveNavigation']}. Finally, the controller sends control actions to the low-level controller of Shmoobot as described in Sec. \ref{['sec:Experiment']}. On the bottom-right corner, we show the schematic of Shmoobot and the necessary frames and notations.
• Figure 3: Interactive Navigation Schematic: First, the robot approaches the object that obstructs the planned path, followed by adaptive pushing towards the desired placement generated by the object planner. Some infeasible placement positions are shown respectively; 1: collision with the map and 2: collision with the planned path. If the object planner concludes that the object is not manipulable, the object is registered as an obstacle in the map and the robot re-plans with the updated map.
• Figure 4: Adaptive Dynamics Accuracy: We compared the accuracy of the adaptive dynamics model $O$ prediction before and after learning on 8 objects whose dynamics were not seen during training. ($m$ is the object's mass, $\mu$ is the planar friction, $\sigma$ is the wheel's rotational friction and the red block on the data icon indicates a locked wheel). The left side is a visualization of the model roll-out of the different horizons, the interaction force is visualized in blue. We further compared with other models discussed in Sec. \ref{['sec:Validation']} of which we listed the prediction MSE at the bottom.
• Figure 5: Object Planner: We show three planned object goals (blue) for the unregistered obstructing object (green), corresponding to a wheelchair with the right wheel locked, a cart with the front-left wheel locked, and a box. The green line is the global path planned by the collision-free navigation planner, the green box is the object's current position and the blue box is the desired goal planner by the object planner.