Fine-grained text-driven dual-human motion generation via dynamic hierarchical interaction

TL;DR

This work targets the challenge of generating fine-grained dual-human motions guided by textual prompts, addressing the dynamic and hierarchical nature of human interaction. It introduces FineDual, a diffusion-based framework with three stages: Self-Learning to decompose overall prompts into per-person instructions and align text-motion at the individual level; Adaptive Adjustment to model inter-person interactions via an interaction distance predictor and an interaction-aware graph; and Teacher-Guided Refinement to synthesize cohesive, fine-grained motions under overall prompt guidance. The approach demonstrates state-of-the-art performance on InterHuman and InterX datasets, with strong improvements in both general motion quality and interaction fidelity, validated through quantitative metrics, ablations, and user studies. The work advances text-driven dual-human motion generation by explicitly leveraging distance information and hierarchical processing, enabling more realistic and instruction-compliant social interactions in synthetic motion data.

Abstract

Human interaction is inherently dynamic and hierarchical, where the dynamic refers to the motion changes with distance, and the hierarchy is from individual to inter-individual and ultimately to overall motion. Exploiting these properties is vital for dual-human motion generation, while existing methods almost model human interaction temporally invariantly, ignoring distance and hierarchy. To address it, we propose a fine-grained dual-human motion generation method, namely FineDual, a tri-stage method to model the dynamic hierarchical interaction from individual to inter-individual. The first stage, Self-Learning Stage, divides the dual-human overall text into individual texts through a Large Language Model, aligning text features and motion features at the individual level. The second stage, Adaptive Adjustment Stage, predicts interaction distance by an interaction distance predictor, modeling human interactions dynamically at the inter-individual level by an interaction-aware graph network. The last stage, Teacher-Guided Refinement Stage, utilizes overall text features as guidance to refine motion features at the overall level, generating fine-grained and high-quality dual-human motion. Extensive quantitative and qualitative evaluations on dual-human motion datasets demonstrate that our proposed FineDual outperforms existing approaches, effectively modeling dynamic hierarchical human interaction.

Figures (7)

• Figure 1: We propose FineDual to generate high-quality dual-human motions with fine-grained interactions. Here, we present four text prompts along with their corresponding motions. Each color represents an individual. The arrow represents the time axes.
• Figure 2: Overview of Our FineDual. The dynamic hierarchical interaction models human interactions at progressively deeper levels. Given the text prompt $c$, our method refines motion from noisy data through T steps, with each stage addressing a distinct level of interaction understanding. Taking step $t$ as an example: (1) The Self-Learning Stage captures individual motion features for each person separately based on their own text prompts, producing $X_{m1}^{s1}$ and $X_{m2}^{s1}$. (2) In the Adaptive Adjustment Stage, the model builds an interaction-aware graph to assess the relationship between individuals, predicting interaction strength and assigning weights to adjust motion features, resulting in interaction-refined outputs $X_{m1}^{s2}$ and $X_{m2}^{s2}$. (3) The Teacher-Guided Refinement Stage synthesizes these individual and interaction-level features with the overall prompt context, generating keyframe-adjusted, final motion outputs $X_{m1}^{s3}$ and $X_{m2}^{s3}$ that represent cohesive and contextually appropriate interactions.
• Figure 3: Visual results compared with existing methods. The arrows represent the time axes.
• Figure 4: Qualitative comparison of different stages. Stage 1, stage 2, and stage 3 represent Self-Learning Stage, Adaptive Adjustment Stage, and Teacher-Guided Refinement Stage respectively. The arrows represent the time axes.
• Figure 5: Qualitative analysis on the coarse-grained interaction problem. The arrows represent the time axes.