RoboDuet: Learning a Cooperative Policy for Whole-body Legged Loco-Manipulation

TL;DR

RoboDuet tackles the challenge of legged loco-manipulation by decoupling control into two cooperative policies: a loco policy for movement and an arm policy for 6D end-effector tracking. The training proceeds in two stages to first establish robust locomotion and then enable whole-body coordination, with reward adjustments facilitating a smooth transition. The framework achieves at least 23% improvement in task success over baselines, supports zero-shot transfer across morphologically similar quadrupeds, and demonstrates strong real-world performance in extreme pose tracking and transfer tasks. This work advances practical whole-body loco-manipulation by enhancing coordination, generalization, and hardware adaptability, while outlining paths for integrating higher-level planning and handling more complex terrains.

Abstract

Fully leveraging the loco-manipulation capabilities of a quadruped robot equipped with a robotic arm is non-trivial, as it requires controlling all degrees of freedom (DoFs) of the quadruped robot to achieve effective whole-body coordination. In this letter, we propose a novel framework RoboDuet, which employs two collaborative policies to realize locomotion and manipulation simultaneously, achieving whole-body control through mutual interactions. Beyond enabling large-range 6D pose tracking for manipulation, we find that the two-policy framework supports zero-shot transfer across quadruped robots with similar morphology and physical dimensions in the real world. Our experiments demonstrate that RoboDuet achieves a 23% improvement in success rate over the baseline in challenging loco-manipulation tasks employing whole-body control. To support further research, we provide open-source code and additional videos on our website: locomanip-duet.github.io.

Figures (7)

• Figure 1: RoboDuet is a framework that affords loco-manipulation and zero-shot transfer on morphologically and dimensionally similar quadruped robots. Top row: zero-shot transfer. This feature enables control across six configurations generated by the permutation of two robotic arms and three quadruped robots, allowing for the replacement of quadruped robots without the need to retrain the entire system. Middle row: loco-manipulation. From left to right, the robot walks to pick up a small ball on grass, grasps a doll from a high table in a café, grabs a bottle on lower stairs, and picks up a cup from an office desk. Bottom row: whole-body control. Given multiple target end-effector poses, the robot can adjust its entire body posture while moving to reach the desired targets and maintain stability.
• Figure 2: Overview of RoboDuet. In Stage 1, the loco policy is trained with fixed robotic arm, enabling the quadruped robot to achieve robust locomotion. In Stage 2, the loco and arm policies are trained simultaneously in a cooperative manner. The loco policy from stage 1 is reused, but the original body orientation commands are replaced by guidance signals $a_t^{arm^G}$ generated by the arm policy, enabling coordinated whole-body control. During the transition between the two stages, reward adjustment is introduced to effectively leverage locomotion priors, facilitating seamless integration for whole-body loco-manipulation.
• Figure 3: The robot system, consisting of a quadruped robot (Unitree Go1 Edu) with a mounted robotic arm (ARX5) and a camera (RealSense D435i).
• Figure 4: Comparison of Solvability and Workspace between FB+IK and RoboDuet.
• Figure 5: Transferring objects between different heights, including the ground (0 cm), a camping chair (20 cm), a cabinet (60 cm), and the cup holder of a standing desk (100 cm). The right part illustrates how RoboDuet utilizes whole-body control to adapt its posture for varying grasp poses during transfer tasks.