Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Human-Humanoid Coordination for Collaborative Object Carrying

Yushi Du, Yixuan Li, Baoxiong Jia, Yutang Lin, Pei Zhou, Wei Liang, Yanchao Yang, Siyuan Huang

TL;DR

This work tackles the challenge of compliant, generalizable human–humanoid collaboration for collaborative object carrying. It introduces COLA, a three-step residual learning framework that first trains a base whole-body controller, then a residual teacher in a closed-loop environment with privileged object states, and finally distills this knowledge into a proprioception-only student capable of deployment without external sensors. In simulation and real-world tests, COLA significantly reduces human effort and achieves accurate, robust coordination across diverse objects and terrains, with COLA-L and COLA-F variants showing improved performance and compliance, respectively. The approach leverages implicit learning of interaction forces via joint-state-target offsets and object dynamics, offering a practical pathway toward real-world deployments in healthcare, domestic assistance, and manufacturing.

Abstract

Human-humanoid collaboration shows significant promise for applications in healthcare, domestic assistance, and manufacturing. While compliant robot-human collaboration has been extensively developed for robotic arms, enabling compliant human-humanoid collaboration remains largely unexplored due to humanoids' complex whole-body dynamics. In this paper, we propose a proprioception-only reinforcement learning approach, COLA, that combines leader and follower behaviors within a single policy. The model is trained in a closed-loop environment with dynamic object interactions to predict object motion patterns and human intentions implicitly, enabling compliant collaboration to maintain load balance through coordinated trajectory planning. We evaluate our approach through comprehensive simulator and real-world experiments on collaborative carrying tasks, demonstrating the effectiveness, generalization, and robustness of our model across various terrains and objects. Simulation experiments demonstrate that our model reduces human effort by 24.7%. compared to baseline approaches while maintaining object stability. Real-world experiments validate robust collaborative carrying across different object types (boxes, desks, stretchers, etc.) and movement patterns (straight-line, turning, slope climbing). Human user studies with 23 participants confirm an average improvement of 27.4% compared to baseline models. Our method enables compliant human-humanoid collaborative carrying without requiring external sensors or complex interaction models, offering a practical solution for real-world deployment.

Learning Human-Humanoid Coordination for Collaborative Object Carrying

TL;DR

This work tackles the challenge of compliant, generalizable human–humanoid collaboration for collaborative object carrying. It introduces COLA, a three-step residual learning framework that first trains a base whole-body controller, then a residual teacher in a closed-loop environment with privileged object states, and finally distills this knowledge into a proprioception-only student capable of deployment without external sensors. In simulation and real-world tests, COLA significantly reduces human effort and achieves accurate, robust coordination across diverse objects and terrains, with COLA-L and COLA-F variants showing improved performance and compliance, respectively. The approach leverages implicit learning of interaction forces via joint-state-target offsets and object dynamics, offering a practical pathway toward real-world deployments in healthcare, domestic assistance, and manufacturing.

Abstract

Human-humanoid collaboration shows significant promise for applications in healthcare, domestic assistance, and manufacturing. While compliant robot-human collaboration has been extensively developed for robotic arms, enabling compliant human-humanoid collaboration remains largely unexplored due to humanoids' complex whole-body dynamics. In this paper, we propose a proprioception-only reinforcement learning approach, COLA, that combines leader and follower behaviors within a single policy. The model is trained in a closed-loop environment with dynamic object interactions to predict object motion patterns and human intentions implicitly, enabling compliant collaboration to maintain load balance through coordinated trajectory planning. We evaluate our approach through comprehensive simulator and real-world experiments on collaborative carrying tasks, demonstrating the effectiveness, generalization, and robustness of our model across various terrains and objects. Simulation experiments demonstrate that our model reduces human effort by 24.7%. compared to baseline approaches while maintaining object stability. Real-world experiments validate robust collaborative carrying across different object types (boxes, desks, stretchers, etc.) and movement patterns (straight-line, turning, slope climbing). Human user studies with 23 participants confirm an average improvement of 27.4% compared to baseline models. Our method enables compliant human-humanoid collaborative carrying without requiring external sensors or complex interaction models, offering a practical solution for real-world deployment.
Paper Structure (22 sections, 5 equations, 5 figures, 4 tables)

This paper contains 22 sections, 5 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Robot-human Collaboration
  4. Compliant Whole-body Control
  5. Methodology
  6. Overview
  7. Whole-body Control Policy
  8. Residual Teacher Policy
  9. Knowledge Distillation
  10. Implementation Details
  11. Training Setup
  12. Observation Space Details
  13. Closed-loop Training Environment
  14. Experiments
  15. Overview
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Overview of COLA. Our Policy mainly consists of three steps: (i) We train a base whole-body control policy to provide a robust whole-body controller. (ii) In the closed-loop training environment, we train a residual teacher policy on top of the whole-body control policy with privileged information for human-humanoid collaboration. (iii) We distill the knowledge from the teacher policy into a student policy for real-world deployment using behavioral cloning.
  • Figure 2: Closed-loop Training Environment. This figure illustrates our closed-loop training environment in simulation. The green arrow represents the goal velocity of the carried object, while the red arrow indicates its current velocity.
  • Figure 3: Qualitative visualizations of human-humanoid collaboration using COLA. This figure showcases our model’s ability to carry diverse objects and perform collaborative skills such as horizontal-velocity and height tracking, including under challenging conditions such as sloped terrains. The interacted objects include a $3$kg rod, an $8$kg box, an $11$kg stretcher, and a $20$kg cart, demonstrating both versatility and generalizability of the proposed method.
  • Figure 4: Quantitative Results on the Effectiveness of Collaborative Carrying. (a) Illustrates the robot's velocity with a force applied to the palm of the robot where the force is linearly increased throughout a time sequence of $10.0$ seconds. (b) Illustrates the height of the robot's pelvis over time when applying external forces on the end-effector of the robot. The solid lines indicate the robot’s height in the simulator under $10N$ external forces on each palm, while dashed lines show the height under $20N$ forces. (c) Illustrates the minimal force required to move the robot in the real world. (d) Illustrates the height difference between the human-held end and the humanoid-held end of the object in real-world experiments.
  • Figure 5: Movement Analysis. When a continuous external force is applied to the robot’s torso, it resists to maintain a stable stance. In contrast, when a smaller force is applied to the robot’s end-effector, it tends to follow the force.