A Framework for Deploying Learning-based Quadruped Loco-Manipulation

TL;DR

The paper presents an open, end-to-end framework for training, benchmarking, and deploying reinforcement-learning–based loco-manipulation on the Unitree B1+Z1 platform, enabling robust sim-to-sim and sim-to-real transfer via a ROS-based stack and a MuJoCo hardware interface. It reveals cross-simulator differences in contact modeling that affect policy behavior and demonstrates that coordinated whole-body control enhances manipulation reach and stability in real-world teleoperation. Key contributions include a unified deployment pipeline, sim2sim and sim2real implementations, and a foot-contact estimation mechanism that supports safe real-time operation. The findings underscore the importance of contact modeling robustness and present mid-pose configurations as a practical guideline for balancing locomotion stability and manipulation reach in real-world deployments. Overall, the work advances reproducible RL-based loco-manipulation on legged mobile manipulators and lays groundwork for open, comparative evaluation in future studies.

Abstract

Quadruped mobile manipulators offer strong potential for agile loco-manipulation but remain difficult to control and transfer reliably from simulation to reality. Reinforcement learning (RL) shows promise for whole-body control, yet most frameworks are proprietary and hard to reproduce on real hardware. We present an open pipeline for training, benchmarking, and deploying RL-based controllers on the Unitree B1 quadruped with a Z1 arm. The framework unifies sim-to-sim and sim-to-real transfer through ROS, re-implementing a policy trained in Isaac Gym, extending it to MuJoCo via a hardware abstraction layer, and deploying the same controller on physical hardware. Sim-to-sim experiments expose discrepancies between Isaac Gym and MuJoCo contact models that influence policy behavior, while real-world teleoperated object-picking trials show that coordinated whole-body control extends reach and improves manipulation over floating-base baselines. The pipeline provides a transparent, reproducible foundation for developing and analyzing RL-based loco-manipulation controllers and will be released open source to support future research.

Figures (16)

• Figure 1: Wholebody control motion deployed on Real-world robot hardware
• Figure 2: Unitree B1 and Z1 platform setup used for development of the framework.
• Figure 3: Inputs (proprioception, previous action, phase cues, command, extrinsics) pass through a 3-layer MLP to produce 12 leg joint targets. A privileged encoder (training) and an adaptation module (deployment) provide $z_t$ for dynamics variation.
• Figure 4: Deployment architecture of the proposed system. The same policy and controller stack run across both real and simulated environments, with only the hardware interface replaced.
• Figure 5: Simulation environments used for sim2sim loco-manipulation experiments: Isaac Gym (Left) and Mujoco (Right).