Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-Speed and Impact Resilient Teleoperation of Humanoid Robots

Sylvain Bertrand, Luigi Penco, Dexton Anderson, Duncan Calvert, Valentine Roy, Stephen McCrory, Khizar Mohammed, Sebastian Sanchez, Will Griffith, Steve Morfey, Alexis Maslyczyk, Achintya Mohan, Cody Castello, Bingyin Ma, Kartik Suryavanshi, Patrick Dills, Jerry Pratt, Victor Ragusila, Brandon Shrewsbury, Robert Griffin

TL;DR

The paper tackles the challenge of achieving high-transparency, low-latency teleoperation for humanoid robots. It introduces an integrated pipeline that combines calibration-free motion capture using seven IMUs, a high-bandwidth kinematics streaming toolbox (KST) with state estimation and predictive IK, a momentum-based whole-body controller, and cycloidal actuators to enable high-speed and impact-resilient operation. The approach achieves a real-time control bandwidth from relatively low-rate input data and demonstrates Nadia performing fast, synchronized tasks such as ping-pong and punching, with latency around 70 ms and robust handling of impacts. This work reduces user setup complexity, enhances safety and responsiveness, and provides open-source access to reproduce and extend the system for real-world, dynamic teleoperation scenarios.

Abstract

Teleoperation of humanoid robots has long been a challenging domain, necessitating advances in both hardware and software to achieve seamless and intuitive control. This paper presents an integrated solution based on several elements: calibration-free motion capture and retargeting, low-latency fast whole-body kinematics streaming toolbox and high-bandwidth cycloidal actuators. Our motion retargeting approach stands out for its simplicity, requiring only 7 IMUs to generate full-body references for the robot. The kinematics streaming toolbox, ensures real-time, responsive control of the robot's movements, significantly reducing latency and enhancing operational efficiency. Additionally, the use of cycloidal actuators makes it possible to withstand high speeds and impacts with the environment. Together, these approaches contribute to a teleoperation framework that offers unprecedented performance. Experimental results on the humanoid robot Nadia demonstrate the effectiveness of the integrated system.

High-Speed and Impact Resilient Teleoperation of Humanoid Robots

TL;DR

The paper tackles the challenge of achieving high-transparency, low-latency teleoperation for humanoid robots. It introduces an integrated pipeline that combines calibration-free motion capture using seven IMUs, a high-bandwidth kinematics streaming toolbox (KST) with state estimation and predictive IK, a momentum-based whole-body controller, and cycloidal actuators to enable high-speed and impact-resilient operation. The approach achieves a real-time control bandwidth from relatively low-rate input data and demonstrates Nadia performing fast, synchronized tasks such as ping-pong and punching, with latency around 70 ms and robust handling of impacts. This work reduces user setup complexity, enhances safety and responsiveness, and provides open-source access to reproduce and extend the system for real-world, dynamic teleoperation scenarios.

Abstract

Teleoperation of humanoid robots has long been a challenging domain, necessitating advances in both hardware and software to achieve seamless and intuitive control. This paper presents an integrated solution based on several elements: calibration-free motion capture and retargeting, low-latency fast whole-body kinematics streaming toolbox and high-bandwidth cycloidal actuators. Our motion retargeting approach stands out for its simplicity, requiring only 7 IMUs to generate full-body references for the robot. The kinematics streaming toolbox, ensures real-time, responsive control of the robot's movements, significantly reducing latency and enhancing operational efficiency. Additionally, the use of cycloidal actuators makes it possible to withstand high speeds and impacts with the environment. Together, these approaches contribute to a teleoperation framework that offers unprecedented performance. Experimental results on the humanoid robot Nadia demonstrate the effectiveness of the integrated system.
Paper Structure (26 sections, 24 equations, 8 figures)

This paper contains 26 sections, 24 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Contributions
  3. Overview
  4. Motion Retargeting
  5. Pelvis retargeting
  6. Hands retargeting
  7. Center of mass retargeting
  8. Footstep streaming
  9. Kinematics streaming toolbox
  10. Filter and safety limits
  11. Bounding box
  12. Rate of change and velocity limits
  13. State estimation and prediction
  14. First-order estimation
  15. Feedback controller-based estimation
  16. ...and 11 more sections

Figures (8)

  • Figure 1: The humanoid robot Nadia being teleoperated by a human operator equipped with a VR headset, four VR trackers, and two VR controllers. Thanks to our kinematics streaming framework and the integration of cycloidal actuators, Nadia can punch a 100 lbs punching bag at human speed.
  • Figure 2: Flowchart of the proposed teleoperation system.
  • Figure 3: Nadia's cycloidal actuators and internal gear mechanism.
  • Figure 4: Performance specifications of cycloidal actuators in the robot's arms. The table details the range of motion (RoM), speed at 48V and 72V, peak torque, and continuous torque for each joint in the arm cycloidActuator2024acycloidActuator2024b.
  • Figure 5: Comparison of VR input, desired, and current poses and velocity for the left robot hand during repeated left punches in free motion.
  • ...and 3 more figures