HeLoM: Hierarchical Learning for Whole-Body Loco-Manipulation in Hexapod Robot

TL;DR

Pushing large objects with hexapod robots requires stable, coordinated whole-body interaction. HeLoM addresses this with a hierarchical RL structure: a Pushing Planner provides high-level foreleg and base commands, while a DreamWaQ-based Controller executes full-body joint actions to maintain balance and apply continuous pushing forces. The system is trained in simulation with curriculum learning and domain randomization and is deployed on a real hexapod without fine-tuning, demonstrating strong performance across objects of varying size and unknown properties. This work advances robust, real-world multi-contact loco-manipulation on multi-legged platforms and offers a scalable approach for sim-to-real transfer in complex manipulation tasks.

Abstract

Robots in real-world environments are often required to move/manipulate objects comparable in weight to their own bodies. Compared to grasping and carrying, pushing provides a more straightforward and efficient non-prehensile manipulation strategy, avoiding complex grasp design while leveraging direct contact to regulate an object's pose. Achieving effective pushing, however, demands both sufficient manipulation forces and the ability to maintain stability, which is particularly challenging when dealing with heavy or irregular objects. To address these challenges, we propose HeLoM, a learning-based hierarchical whole-body manipulation framework for a hexapod robot that exploits coordinated multi-limb control. Inspired by the cooperative strategies of multi-legged insects, our framework leverages redundant contact points and high degrees of freedom to enable dynamic redistribution of contact forces. HeLoM's high-level planner plans pushing behaviors and target object poses, while its low-level controller maintains locomotion stability and generates dynamically consistent joint actions. Our policies trained in simulation are directly deployed on real robots without additional fine-tuning. This design allows the robot to maintain balance while exerting continuous and controllable pushing forces through coordinated foreleg interaction and supportive hind-leg propulsion. We validate the effectiveness of HeLoM through both simulation and real-world experiments. Results show that our framework can stably push boxes of varying sizes and unknown physical properties to designated goal poses in the real world.

Figures (7)

• Figure 1: Examples of the modular hexapod robot pushes different objects to desired poses using its forelegs with our proposed HeLoM framework, shown in simulation (top) and real-world experiments (bottom).
• Figure 2: Overview of the proposed HeLoM framework. Dashed arrows indicate components used only in simulation, while solid arrows represent components used in real-world deployment. During training, we first pre-train the Controller with randomly sampled commands to establish loco-manipulation ability. Subsequently, the network parameters of Controller are frozen, and Planner is trained on top of it to learn effective pushing behaviour commands. To enhance smoothness and coordination, Planner and Controller are executed synchronously at 50 Hz.
• Figure 3: Three representative pushing scenarios in simulation. A and D show the top and side views of the pushing process for Box 1, respectively; B and E for Box 2; and C and F for Box 3. The desired box poses are highlighted in green in A–C. G summarizes the object properties and test target configurations. Here, $\Delta\theta_z\textbf{(deg)}$ denotes the angular difference between the object’s initial orientation and the desired orientation (z-axis), while $\Delta {p}_{obj,x}$ and $\Delta {p}_{obj,y}$ represent the distance from the object. In addition, we define the robot’s initial heading as the positive $x$-axis of the robot coordinate system, and the left-hand side as the positive $y$-axis. All subsequent descriptions are based on this coordinate system.
• Figure 4: Tracking performance of HeLoM and HeLoM w/o smoothness loss with respect to desired velocity and foot position, specifically illustrating tracking of velocity in the $x$-axis and position in the $z$-axis.
• Figure 5: Complete process of robot–object interaction in an experimental trial. A foot marked with a white dot indicates no contact, while other colors denote contact with the corresponding surfaces and display the contact-force vector.