HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit

TL;DR

HOMIE introduces a unified humanoid loco-manipulation teleoperation cockpit that fuses a reinforcement-learning body policy mapped to a pedal with isomorphic exoskeleton arms and motion-sensing gloves, enabling fast, accurate full-body control at low cost. Key innovations include an upper-body pose curriculum, height-tracking reward, and symmetry-based data augmentation, plus hardware design that eliminates inverse kinematics via direct joint mapping. The approach generalizes across robot morphologies, enables real-world and simulated operation, and supports autonomous policy training from teleoperation data. Demonstrations show significantly faster task completion, robust real-world performance, and viable data reuse for imitation learning, with open-source availability and a $0.5k hardware package. These results suggest a practical path toward accessible, generalizable humanoid loco-manipulation in realistic environments.

Abstract

Generalizable humanoid loco-manipulation poses significant challenges, requiring coordinated whole-body control and precise, contact-rich object manipulation. To address this, this paper introduces HOMIE, a semi-autonomous teleoperation system that combines a reinforcement learning policy for body control mapped to a pedal, an isomorphic exoskeleton arm for arm control, and motion-sensing gloves for hand control, forming a unified cockpit to freely operate humanoids and establish a data flywheel. The policy incorporates novel designs, including an upper-body pose curriculum, a height-tracking reward, and symmetry utilization. These features enable the system to perform walking and squatting to specific heights while seamlessly adapting to arbitrary upper-body poses. The exoskeleton, by eliminating the reliance on inverse dynamics, delivers faster and more precise arm control. The gloves utilize Hall sensors instead of servos, allowing even compact devices to achieve 15 or more degrees of freedom and freely adapt to any model of dexterous hands. Compared to previous teleoperation systems, HOMIE stands out for its exceptional efficiency, completing tasks in half the time; its expanded working range, allowing users to freely reach high and low areas as well as interact with any objects; and its affordability, with a price of just $500. The system is fully open-source, demos and code can be found in our https://homietele.github.io/.

Figures (21)

• Figure 1: System Overview.(a): how an operator uses the exoskeleton-based hardware system to control humanoid robots in the real world and simulation. (b): how $\pi_{loco}$ controls the robots, the data collection process for training $\pi_{auto}$, and how $\pi_{auto}$ takes over the operator to control the robots. Communication between the cockpit and the robot is achieved via Wi-Fi.
• Figure 2: RL training framework of HOMIE.
• Figure 3: Different robots are trained to walk and squat with continuous changing upper-body poses in Isaac Gym.
• Figure 4: Upper-body Exoskeleton.(a): The model architecture and physical demonstration of motion-sensing gloves. (b): The structural architecture of the upper-body exoskeleton system, comprising an isomorphic exoskeleton and motion-sensing gloves, with kinematic mapping methodology between the isomorphic exoskeleton and Unitree G1.
• Figure 5: Pedal command control. The three small pedals respectively control $[0, \pm{\omega_{max}}]$, $[{H_{min}}, {H_{max}}]$, and $[0, \pm{V_{max}}]$. The left-side switching button is used to toggle between left and right modes, while the right-side switching button is used to toggle between forward and backward modes.