HeteroMorpheus: Universal Control Based on Morphological Heterogeneity Modeling

TL;DR

Addresses the challenge of universal control across morphologies by learning a single policy $\pi_{\theta}(a_k|s_k,M_k)$ that generalizes across designs. Proposes HeteroMorpheus, a heterogeneous graph Transformer that encodes morphology via node types and edge types to enable targeted message passing. Demonstrates superior generalization, including zero-shot transfer to unseen morphologies, and faster, more stable fine-tuning on manipulation tasks in Evogym, outperforming Amorpheus and MetaMorph. This work advances morphology-aware universal control and points to richer edge definitions and neighbor scopes as future directions.

Abstract

In the field of robotic control, designing individual controllers for each robot leads to high computational costs. Universal control policies, applicable across diverse robot morphologies, promise to mitigate this challenge. Predominantly, models based on Graph Neural Networks (GNN) and Transformers are employed, owing to their effectiveness in capturing relational dynamics across a robot's limbs. However, these models typically employ homogeneous graph structures that overlook the functional diversity of different limbs. To bridge this gap, we introduce HeteroMorpheus, a novel method based on heterogeneous graph Transformer. This method uniquely addresses limb heterogeneity, fostering better representation of robot dynamics of various morphologies. Through extensive experiments we demonstrate the superiority of HeteroMorpheus against state-of-the-art methods in the capability of policy generalization, including zero-shot generalization and sample-efficient transfer to unfamiliar robot morphologies.

Figures (5)

• Figure 1: The Overall Architecture of HeteroMorpheus. We illustrate the action generation process for a voxel node in the robot. The local observations of the target node $t$ and its neighboring nodes $s_1$ and $s_2$ are linearly embedded based on their corresponding node types. The embedded vectors are then added to the learnt positional embedding as the initial features of the node, which are subsequently fed into the HGT for heterogeneous attention calculation, message passing, and aggregation. The output of HGT is concatenated with the linear embedding of the global observation and passed to the Decoder to generate a distribution over actions.
• Figure 2: Evogym tasks. In Walker-v0, the robot's objective is to travel as far as possible on flat terrain. Carrier-v0 involves transporting an object to maximize distance covered. In Pusher-v0, the aim is to push an object to achieve the greatest displacement. UpStepper-v0 presents the challenge of ascending stairs of varying lengths to cover maximum distance. In Catcher-v0, the robot needs to receive an object thrown from a height and in a rotating state, and then carry the object forward as much as possible. All tasks measure progress along the positive x-axis.
• Figure 3: The training curves of different methods in each environment. We evaluate on 5 different seeds and plot the mean of average returns over all morphologies. Shaded regions denotes standard deviation.
• Figure 4: Comparison of different method's transfer learning performance on an unknown robot morphology set. The graph presents the results of both zero-shot learning and fine-tuning, obtained through five repeated experiments. Shaded regions denotes standard deviation.
• Figure 5: Attention Matrix Analysis. We plot the stable rank of the attention matrix ($sr(A_0)$) for Pusher-v0, as well as the morphology of the robot and the attention matrix corresponding to the moments when the stabe rank is at its peak and valley. The attention matrix on the left corresponds to the peak, the attention matrix on the right corresponds to the valley. The horizontal and vertical coordinates of the matrix represent the voxel block labels arranged in order from left to right and from top to bottom for the robots.