Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UniBiDex: A Unified Teleoperation Framework for Robotic Bimanual Dexterous Manipulation

Zhongxuan Li, Zeliang Guo, Jun Hu, David Navarro-Alarcon, Jia Pan, Hongmin Wu, Peng Zhou

TL;DR

UniBiDex tackles the need for unified, high‑fidelity teleoperation of dual‑arm dexterous robots by bridging VR and leader‑follower inputs through a single, safety‑aware control stack. It leverages null‑space coordination within a unified kinematics module to generate collision‑free, manipulable dual‑arm configurations during tasks. Empirical results on a long‑horizon kitchen tidying task show higher success rates and smoother trajectories compared with strong baselines, across both input modalities. The authors release hardware and software as open‑source to accelerate data collection for imitation learning and broader adoption in robot learning.

Abstract

We present UniBiDex a unified teleoperation framework for robotic bimanual dexterous manipulation that supports both VRbased and leaderfollower input modalities UniBiDex enables realtime contactrich dualarm teleoperation by integrating heterogeneous input devices into a shared control stack with consistent kinematic treatment and safety guarantees The framework employs nullspace control to optimize bimanual configurations ensuring smooth collisionfree and singularityaware motion across tasks We validate UniBiDex on a longhorizon kitchentidying task involving five sequential manipulation subtasks demonstrating higher task success rates smoother trajectories and improved robustness compared to strong baselines By releasing all hardware and software components as opensource we aim to lower the barrier to collecting largescale highquality human demonstration datasets and accelerate progress in robot learning.

UniBiDex: A Unified Teleoperation Framework for Robotic Bimanual Dexterous Manipulation

TL;DR

UniBiDex tackles the need for unified, high‑fidelity teleoperation of dual‑arm dexterous robots by bridging VR and leader‑follower inputs through a single, safety‑aware control stack. It leverages null‑space coordination within a unified kinematics module to generate collision‑free, manipulable dual‑arm configurations during tasks. Empirical results on a long‑horizon kitchen tidying task show higher success rates and smoother trajectories compared with strong baselines, across both input modalities. The authors release hardware and software as open‑source to accelerate data collection for imitation learning and broader adoption in robot learning.

Abstract

We present UniBiDex a unified teleoperation framework for robotic bimanual dexterous manipulation that supports both VRbased and leaderfollower input modalities UniBiDex enables realtime contactrich dualarm teleoperation by integrating heterogeneous input devices into a shared control stack with consistent kinematic treatment and safety guarantees The framework employs nullspace control to optimize bimanual configurations ensuring smooth collisionfree and singularityaware motion across tasks We validate UniBiDex on a longhorizon kitchentidying task involving five sequential manipulation subtasks demonstrating higher task success rates smoother trajectories and improved robustness compared to strong baselines By releasing all hardware and software components as opensource we aim to lower the barrier to collecting largescale highquality human demonstration datasets and accelerate progress in robot learning.
Paper Structure (16 sections, 7 equations, 6 figures, 1 table)

This paper contains 16 sections, 7 equations, 6 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Teleoperation Devices
  4. Kinematics Constraints in Bimanual Manipulation
  5. The Methodology
  6. Overall System Design
  7. Input Preprocessing and Motion Retargeting
  8. Notation
  9. Relative Motion Retargeting
  10. Bimanual Teleoperation Coordination
  11. Haptic Feedback
  12. Experiments
  13. User Study Procedure
  14. Teleoperation Results
  15. Failure Case Analysis and Discussion
  16. ...and 1 more sections

Figures (6)

  • Figure 1: We show the UniBiDex framework built for contact-rich bimanual teleoperation. The user can teleoperate the robot arms by controlling either the VR headset or leader arms. The system processes heterogeneous input devices with a unified kinematics module.
  • Figure 2: Illustration of the proposed teleoperation framework. The system processes input commands from heterogeneous devices and formulates task-related and safety constraints for bimanual control as a single optimization problem.
  • Figure 3: (a) VR mode. (b) Leader–follower arm mode. UniBiDex supports both VR- and leader arm-based control with nearly identical performance.
  • Figure 4: Illustration of the optimal reference configuration. The right arm joint configuration in (c) provides greater degrees of freedom compared to the arm in (b), resulting in more compliant control in subsequent motions.
  • Figure 5: Overview of the leader–follower arm teleoperation workflow for the household kitchen–tidying task, showing the five sequential subtasks: (1) item unpacking, (2) shelf organization, (3) towel folding, (4) towel placement, and (5) clamp attachment.
  • ...and 1 more figures