Interact2Ar: Full-Body Human-Human Interaction Generation via Autoregressive Diffusion Models

TL;DR

Interact2Ar tackles the challenge of realistic full-body human-human interaction generation by introducing a text-conditioned autoregressive diffusion model with cooperative, body-part specialized denoisers. Its Mixed Memory memory strategy enables long-context, adaptive generation, supporting tasks such as temporal motion composition, disturbance adaptation, and sequential multi-person interactions. The paper also advances evaluation by building robust body- and hand-specific evaluators and demonstrates state-of-the-art performance on the Inter-X benchmark. Together, these contributions yield more expressive, coherent, and controllable human-human interactions with practical applications in animation and virtual collaboration. The work highlights remaining challenges in body-shape diversity and provides a clear path toward broader, more diverse interaction modeling.

Abstract

Generating realistic human-human interactions is a challenging task that requires not only high-quality individual body and hand motions, but also coherent coordination among all interactants. Due to limitations in available data and increased learning complexity, previous methods tend to ignore hand motions, limiting the realism and expressivity of the interactions. Additionally, current diffusion-based approaches generate entire motion sequences simultaneously, limiting their ability to capture the reactive and adaptive nature of human interactions. To address these limitations, we introduce Interact2Ar, the first end-to-end text-conditioned autoregressive diffusion model for generating full-body, human-human interactions. Interact2Ar incorporates detailed hand kinematics through dedicated parallel branches, enabling high-fidelity full-body generation. Furthermore, we introduce an autoregressive pipeline coupled with a novel memory technique that facilitates adaptation to the inherent variability of human interactions using efficient large context windows. The adaptability of our model enables a series of downstream applications, including temporal motion composition, real-time adaptation to disturbances, and extension beyond dyadic to multi-person scenarios. To validate the generated motions, we introduce a set of robust evaluators and extended metrics designed specifically for assessing full-body interactions. Through quantitative and qualitative experiments, we demonstrate the state-of-the-art performance of Interact2Ar.

Figures (7)

• Figure 1: We introduce Interact2Ar, the first text-conditioned autoregressive diffusion model for generating full-body human-human interactions with detailed hand motions. Our autoregressive model, with a novel memory strategy, enhances the quality of the generated interaction and enables adaptive capabilities, including temporal composition, adaptation to disturbances, and multi-person scenarios.
• Figure 2: A) Multi-Head Denoiser: An encoder feeds noised motion and condition information to specialized heads for body, trajectory, and hands denoising. B) Cooperative Denoisers: Parallel streams with shared weights generate each interactant while cross-attention shares inter-personal information. C) Autoregressive: Sequential generation of sub-motions conditioned on previously generated frames.
• Figure 3: Mixed Memory enables access to both detailed short-term information, facilitating seamless transitions, along with long-term context, avoiding action repetition in long interactions. Our proposed Mixed Memory overcomes the limitations of regular context memory, providing up to a $\times$3 reduction in memory size.
• Figure 4: User study with the average ranking of 35 participants evaluating the text alignment and hand quality of 10 interactions.
• Figure 5: Interact2Ar comparison with SOTA. Our method Interac2Ar (top) generates higher-quality interactions with improved alignment to textual descriptions and more realistic hand motions in comparison to the previous SOTA InterMask (bottom).