Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OpenHEART: Opening Heterogeneous Articulated Objects with a Legged Manipulator

Seonghyeon Lim, Hyeonwoo Lee, Seunghyun Lee, I Made Aswin Nahrendra, Hyun Myung

TL;DR

Sampling-based Abstracted Feature Extraction (SAFE), which encodes handle and panel geometry into a compact low-dimensional representation, improving cross-domain generalization and Articulation Information Estimator is introduced to adaptively mix proprioception with exteroception to estimate opening direction and range of motion for each object.

Abstract

Legged manipulators offer high mobility and versatile manipulation. However, robust interaction with heterogeneous articulated objects, such as doors, drawers, and cabinets, remains challenging because of the diverse articulation types of the objects and the complex dynamics of the legged robot. Existing reinforcement learning (RL)-based approaches often rely on high-dimensional sensory inputs, leading to sample inefficiency. In this paper, we propose a robust and sample-efficient framework for opening heterogeneous articulated objects with a legged manipulator. In particular, we propose Sampling-based Abstracted Feature Extraction (SAFE), which encodes handle and panel geometry into a compact low-dimensional representation, improving cross-domain generalization. Additionally, Articulation Information Estimator (ArtIEst) is introduced to adaptively mix proprioception with exteroception to estimate opening direction and range of motion for each object. The proposed framework was deployed to manipulate various heterogeneous articulated objects in simulation and real-world robot systems. Videos can be found on the project website: https://openheart-icra.github.io/OpenHEART/

OpenHEART: Opening Heterogeneous Articulated Objects with a Legged Manipulator

TL;DR

Sampling-based Abstracted Feature Extraction (SAFE), which encodes handle and panel geometry into a compact low-dimensional representation, improving cross-domain generalization and Articulation Information Estimator is introduced to adaptively mix proprioception with exteroception to estimate opening direction and range of motion for each object.

Abstract

Legged manipulators offer high mobility and versatile manipulation. However, robust interaction with heterogeneous articulated objects, such as doors, drawers, and cabinets, remains challenging because of the diverse articulation types of the objects and the complex dynamics of the legged robot. Existing reinforcement learning (RL)-based approaches often rely on high-dimensional sensory inputs, leading to sample inefficiency. In this paper, we propose a robust and sample-efficient framework for opening heterogeneous articulated objects with a legged manipulator. In particular, we propose Sampling-based Abstracted Feature Extraction (SAFE), which encodes handle and panel geometry into a compact low-dimensional representation, improving cross-domain generalization. Additionally, Articulation Information Estimator (ArtIEst) is introduced to adaptively mix proprioception with exteroception to estimate opening direction and range of motion for each object. The proposed framework was deployed to manipulate various heterogeneous articulated objects in simulation and real-world robot systems. Videos can be found on the project website: https://openheart-icra.github.io/OpenHEART/
Paper Structure (21 sections, 1 equation, 7 figures, 3 tables)

This paper contains 21 sections, 1 equation, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. SAFE: Sampling-based Abstracted Feature Extraction
  5. ArtIEst: Articulation Information Estimator
  6. Exteroception-based Estimator
  7. Proprioception-augmented Estimator
  8. Belief Gating Mechanism
  9. High-level Actor
  10. High-level Actor Observation
  11. Reward Function
  12. Experiments
  13. Experiment Setup
  14. Compared Methods
  15. Efficiency of the Proposed Representation
  16. ...and 6 more sections

Figures (7)

  • Figure 1: We present a framework for opening heterogeneous articulated objects with a legged manipulator. The robot can open diverse articulated objects, such as (a) a revolute cabinet with a vertical handle and (b) a prismatic drawer with a horizontal handle without any object-specific models.
  • Figure 2: The framework is hierarchically structured, with a high-level planner and (a) a low-level controller. SAFE is introduced to efficiently represent the object shape, while mitigating overfitting. (b) ArtIEst estimates the articulation information by adaptively mixing exteroception and proprioception. The history encoder extracts features from proprioception history.
  • Figure 3: Defined articulation information (pink solid arrow) for objects with (a) a revolute joint and (b) a prismatic joint. The expected opening trajectory is shown as a blue dashed arrow.
  • Figure 4: Demonstration of a heterogeneous articulated object opening in the simulation. The objects differ in their handle shape, dimension, and opening direction.
  • Figure 5: (a) Learning curve of opening reward. Ours shows the highest performance compared with the baselines. Saliency maps are shown for the object with (b) revolute joint and (c) prismatic joint. Object appearances are depicted at the top line, the saliency maps from Point cloud-based policy, and Ours are shown in the middle and below each subfigure.
  • ...and 2 more figures

Theorems & Definitions (1)

  • Remark III.1