Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning

Abstract

Humanoid robotics has strong potential to transform daily service and caregiving applications. Although recent advances in general motion tracking within physics engines (GMT) have enabled virtual characters and humanoid robots to reproduce a broad range of human motions, these behaviors are primarily limited to contact-less social interactions or isolated movements. Assistive scenarios, by contrast, require continuous awareness of a human partner and rapid adaptation to their evolving posture and dynamics. In this paper, we formulate the imitation of closely interacting, force-exchanging human-human motion sequences as a multi-agent reinforcement learning problem. We jointly train partner-aware policies for both the supporter (assistant) agent and the recipient agent in a physics simulator to track assistive motion references. To make this problem tractable, we introduce a partner policies initialization scheme that transfers priors from single-human motion-tracking controllers, greatly improving exploration. We further propose dynamic reference retargeting and contact-promoting reward, which adapt the assistant's reference motion to the recipient's real-time pose and encourage physically meaningful support. We show that AssistMimic is the first method capable of successfully tracking assistive interaction motions on established benchmarks, demonstrating the benefits of a multi-agent RL formulation for physically grounded and socially aware humanoid control.

Figures (10)

• Figure 1: AssistMimic: We propose a multi-agent RL framework capable of learning robust Supporter and Recipient policies from noisy, close-proximity motion sequences. By leveraging single-person motion priors, a novel recipient-adaptive reference retargeting mechanism, and contact-promoting rewards, AssistMimic becomes the first physics-based controller to successfully track such complex, high-contact reference motions. Snapshots are arranged chronologically from left to right.
• Figure 2: Learning contact-rich assistive behaviors is substantially more difficult in the close-contact interactions that we target (bottom) than in contact-less social interactions (top) or isolated motions (gray SR curve). AssistMimic addresses these challenges, achieving successful imitation for the first time, as shown in the improved orange SR curve.
• Figure 3: Overview of AssistMimic. We train tracking-based humanoid control policies for both the recipient and the supporter, optimizing them to imitate a paired reference motion sequence. Our architecture builds on the single-agent tracking framework of PHC Luo_2023_ICCV, extending it with partner-aware state inputs and augmenting standard imitation rewards with recipient-aware reference retargeting and contact-incentivizing reward terms. The policy $\pi_m$ takes as input the proprioceptive state $s_{\text{prior},t}^{(m)}$, the assistive state $s_{\text{assist},t}^{(m)}$ (the interaction context and partner information), and the goal $g_t^{(m)}$ to output the action $a_t^{(m)}$.
• Figure 4: Failure of Kinematic Baselines.
• Figure 5: Qualitative evaluation with HHI-Assist. The red boxes highlight whether the supporter’s hands correctly adjust to the recipient’s position and provide appropriate support.