DoorBot: Closed-Loop Task Planning and Manipulation for Door Opening in the Wild with Haptic Feedback

TL;DR

This paper addresses the generalization gap in door-opening tasks for robots operating in unstructured environments by introducing a haptic-informed closed-loop control framework that fuses vision with real-time haptic feedback. The approach uses a hierarchical controller with a high-level state-machine planner and six low-level motion primitives, augmented by the Grasping-and-Unlocking Model (GUM) to refine grasping and unlocking trajectories. Field experiments on 20 unseen doors demonstrate a 90% average success rate, highlighting the system's robustness and ability to adapt to diverse door types and mechanisms. The work advances open-world articulated-object manipulation by showing how low-cost haptic signals can robustly guide manipulation when visual cues are insufficient and simple data suffice for learning-based components.

Abstract

Robots operating in unstructured environments face significant challenges when interacting with everyday objects like doors. They particularly struggle to generalize across diverse door types and conditions. Existing vision-based and open-loop planning methods often lack the robustness to handle varying door designs, mechanisms, and push/pull configurations. In this work, we propose a haptic-aware closed-loop hierarchical control framework that enables robots to explore and open different unseen doors in the wild. Our approach leverages real-time haptic feedback, allowing the robot to adjust its strategy dynamically based on force feedback during manipulation. We test our system on 20 unseen doors across different buildings, featuring diverse appearances and mechanical types. Our framework achieves a 90% success rate, demonstrating its ability to generalize and robustly handle varied door-opening tasks. This scalable solution offers potential applications in broader open-world articulated object manipulation tasks.

Figures (8)

• Figure 1: We propose a hierarchical closed-loop controller to help a mobile robot automatically open various doors and walk through them in open environments. Our method can robustly generalize to different handles in the wild.
• Figure 2: We design a state machine as the high-level planner. Aside from regular working transitions, we define error-handling transitions to recover from unexpected scenarios. For example, when OPEN_COLLISION is triggered due to collision in the open primitive, the robot then transitions back to grasp.
• Figure 3: Overview of GUM. GUM refines the model-based grasp pose prior for the grasp primitive, and simultaneously predicts the motion trajectory for unlocking the door handle.
• Figure 4: Haptic feedback in 3 motion primitives. For unlock-lever and unlock-knob, the current threshold for the elbow joint tells the robot when to stop. For open the increase/decrease of current feedback on the elbow joint shows the push-/pull-type of the door.
• Figure 5: Visual appearance and configurations of our bimanual mobile robot.