Effective Virtual Reality Teleoperation of an Upper-body Humanoid with Modified Task Jacobians and Relaxed Barrier Functions for Self-Collision Avoidance

Abstract

We present an approach for retartgeting off-the-shelf Virtual Reality (VR) trackers to effectively teleoperate an upper-body humanoid while ensuring self-collision-free motions. Key to the effectiveness was the proper assignment of trackers to joint sets via modified task Jacobians and relaxed barrier functions for self-collision avoidance. The approach was validated on Apptronik's Astro hardware by demonstrating manipulation capabilities on a table-top environment with pick-and-place box packing and a two-handed box pick up and handover task.

Figures (2)

• Figure 1: Four 6 DoF trackers (headset, left controller, right controller, waist tracker) and the set of joints they control on the robot. Using modified task Jacobians, the headset orientation controls the neck joints, the hand controllers' pose control only the arm joints, the waist tracker's vertical axis controls the torso yaw joint, and the forward position of the headset controls the leaning angle of the robot using the torso pitch joint.
• Figure 2: Apptronik's Astro was teleoperated to perform a box packing task and a coordinated two-handed box pickup and handover to a human