Embodiment-Aware Generalist Specialist Distillation for Unified Humanoid Whole-Body Control

TL;DR

The paper tackles cross-embodiment generalization in humanoid whole-body control by proposing EAGLE, an embodiment-aware generalist-specialist distillation framework. EAGLE uses a unified, high-dimensional command interface and an iterative loop that forks embodiment-specific specialists from a generalist, refines them on their robots, and distills the skills back into the generalist using both action and representation-level losses. Empirical results across five simulated humanoids and four real-world robots show improved command-tracking accuracy and robust performance for behaviors beyond walking, including squatting and leaning, with successful zero-shot sim-to-real transfer. This approach demonstrates a scalable path toward fleet-level humanoid control without per-robot reward tuning, enhancing practical deployment potential in diverse morphologies.

Abstract

Humanoid Whole-Body Controllers trained with reinforcement learning (RL) have recently achieved remarkable performance, yet many target a single robot embodiment. Variations in dynamics, degrees of freedom (DoFs), and kinematic topology still hinder a single policy from commanding diverse humanoids. Moreover, obtaining a generalist policy that not only transfers across embodiments but also supports richer behaviors-beyond simple walking to squatting, leaning-remains especially challenging. In this work, we tackle these obstacles by introducing EAGLE, an iterative generalist-specialist distillation framework that produces a single unified policy that controls multiple heterogeneous humanoids without per-robot reward tuning. During each cycle, embodiment-specific specialists are forked from the current generalist, refined on their respective robots, and new skills are distilled back into the generalist by training on the pooled embodiment set. Repeating this loop until performance convergence produces a robust Whole-Body Controller validated on robots such as Unitree H1, G1, and Fourier N1. We conducted experiments on five different robots in simulation and four in real-world settings. Through quantitative evaluations, EAGLE achieves high tracking accuracy and robustness compared to other methods, marking a step toward scalable, fleet-level humanoid control. See more details at https://eagle-wbc.github.io/

Figures (6)

• Figure 1: In this work, we propose a distillation framework that yields a single whole-body controller that runs on heterogeneous humanoids. Through cyclic forking of embodiment-specific specialists and distillation back into the generalist, the resulting policy executes rich commands (walking, squatting, and leaning) in simulation and the real world, outperforming baselines in tracking accuracy and robustness.
• Figure 2: Method Overview.(a) Unified command interface. The command vector $\mathbf c_t$ comprises task commands $\mathbf v_t$ (linear velocities $v_x,v_y$, angular velocity $\omega$) and behavior commands $\mathbf b_t$ (base height $h$, body pitch $p$); together with a window of robot proprioception $s_t$, they form the observation $o_t$. (b) Generalist-specialist distillation framework. Each round copies$\pi_g$ to $N$ specialists $\{\pi_{s_i}\}$ for per-robot fine-tuning, then distills back by running $\pi_g$, relabeling actions with the corresponding specialist, and updating with the loss in Eq. \ref{['eqn:dagger-loss']}. Repeating this loop yields a single controller that scales across embodiments while retaining rich whole-body commands.
• Figure 3: Tracking-error curves for $E_{v_x}$, $E_{v_y}$, and $E_{\omega}$ (left to right). The HTML]FFFBE6$\vartriangle$, $\blacktriangle$ represents the specialist and generalist policy respectively (as described at lines 2-9 of Algo. \ref{['alg:overall']}). We can see the trend that, after each distillation cycle, both symbols move downward, showing that iterative distillation steadily improves tracking accuracy for specialists and generalists.
• Figure 4: t-SNE visualization of policy latent-space representation. By commanding different robots to walk forward for certain timesteps, we can see w/o embodiment-aware observation (left), the policy outputs latent vectors $e_\pi(\cdot)$ collapse into overlapping clusters. In contrast, our method (right) is better separated, showing successful morphology-aware representation.
• Figure 5: Real-world synchronous movement. We also developed a shared low-level control framework to show our policy can let different robots execute synchronous movement, from leaning (II) to squatting (III).