UniDoorManip: Learning Universal Door Manipulation Policy Over Large-scale and Diverse Door Manipulation Environments

TL;DR

This work tackles learning a universal policy for opening doors across diverse categories and mechanisms by introducing a large-scale, part-based door dataset and a realistic IsaacGym-based environment that includes latching mechanisms and partial occlusions. It proposes a three-stage, fully modular policy—handle grasping, handle manipulation, and door opening—trained in reverse order with conditioned training to integrate into a single universal policy. Key contributions include the dataset (328 bodies, 204 handles across 6 categories), the realistic multi-mechanism environment with occlusions and a mobile robot arm, and strong empirical results including real-world transfer that surpass baselines and ablations. The approach advances robust, generalizable door manipulation for embodied agents operating in broad real-world contexts.

Abstract

Learning a universal manipulation policy encompassing doors with diverse categories, geometries and mechanisms, is crucial for future embodied agents to effectively work in complex and broad real-world scenarios. Due to the limited datasets and unrealistic simulation environments, previous works fail to achieve good performance across various doors. In this work, we build a novel door manipulation environment reflecting different realistic door manipulation mechanisms, and further equip this environment with a large-scale door dataset covering 6 door categories with hundreds of door bodies and handles, making up thousands of different door instances. Additionally, to better emulate real-world scenarios, we introduce a mobile robot as the agent and use the partial and occluded point cloud as the observation, which are not considered in previous works while possessing significance for real-world implementations. To learn a universal policy over diverse doors, we propose a novel framework disentangling the whole manipulation process into three stages, and integrating them by training in the reversed order of inference. Extensive experiments validate the effectiveness of our designs and demonstrate our framework's strong performance. Code, data and videos are avaible on https://unidoormanip.github.io/.

Figures (5)

• Figure 1: Our pipeline for the framework. We disentangle the entire door manipulation process into three stages: handle grasping, handle manipulation and door opening. We predict affordance map to graps the handle and employ the similar formulation but train separate policy for handle manipulation and door opening. Besides, We integrate the three policies for each stage leveraging conditioned training.
• Figure 2: Manipulation Sequence Guided by Our Universal Manipulation Policy.
• Figure 3: Comparison of the ablations and ours for different door joint angles. Here for each door joint angle, we do experiments on all categories and get the average success rate.
• Figure 4: Failure cases of ablated versions.
• Figure 5: Real-World Experiments.