HIL: Hybrid Imitation Learning of Diverse Parkour Skills from Videos

TL;DR

Hybrid Imitation Learning (HIL) combines motion tracking and adversarial imitation learning to train a single controller that can reproduce diverse parkour skills while adapting to unseen environments. By training in parallel with a unified scene-centric observation space and employing PSI and ET, HIL achieves broader skill coverage, reduced mode collapse, and competitive task performance compared to baselines. The approach enables natural, context-aware motion across varied obstacles and demonstrates generalization to non-parkour interactions, suggesting broad applicability to physics-based character animation. The combination of a transformer-based policy, scene-conditioned inputs, and a discriminator-driven style reward provides a robust framework for learning versatile, life-like locomotion in complex environments.

Abstract

Recent data-driven methods leveraging deep reinforcement learning have been an effective paradigm for developing controllers that enable physically simulated characters to produce natural human-like behaviors. However, these data-driven methods often struggle to adapt to novel environments and compose diverse skills coherently to perform more complex tasks. To address these challenges, we propose a hybrid imitation learning (HIL) framework that combines motion tracking, for precise skill replication, with adversarial imitation learning, to enhance adaptability and skill composition. This hybrid learning framework is implemented through parallel multi-task environments and a unified observation space, featuring an agent-centric scene representation to facilitate effective learning from the hybrid parallel environments. Our framework trains a unified controller on parkour data sourced from Internet videos, enabling a simulated character to traverse through new environments using diverse and life-like parkour skills. Evaluations across challenging parkour environments demonstrate that our method improves motion quality, increases skill diversity, and achieves competitive task completion compared to previous learning-based methods.

Figures (9)

• Figure 1: Network architectures of the policy, the critic, and the discriminator. The policy takes character state $s_t$, scene point cloud $c_t$, and target goal location $g_t$ as input to output the action. The critic additionally takes a task indicator variable $k_t$ as input. For the discriminator, n-step state transitions $s_{t-n:t}$ and scene point cloud $c_{t-n:t}$ are provided.
• Figure 2: Our controller enables physically simulated characters to perform a wide variety of interactions.
• Figure 3: Motion comparisons with baselines. In this example, 'Task reward' produces unnatural behaviors to clear obstacles as quickly as possible. 'w/o D' struggles to perform appropriate skills for specific obstacles, due to the independent optimization of the two tasks. AMP suffers from severe mode collapse, repeatedly using the same skills across various obstacles. HIL generates more natural and context-aware behaviors with diverse skills.
• Figure 4: Skill coverage comparison. The plots show the frequency of skill usage across 'Task reward', AMP, and our method (HIL). 'Task reward' exhibits significant bias, over-relying on certain skills, while AMP also suffers from mode collapse, with less skill diversity. HIL demonstrates broader skill usage, effectively utilizing a diverse range of skills and achieving more balanced coverage of the reference dataset.
• Figure 5: The character utilizes diverse skills to clear complex obstacle courses. While the reference data includes only individual skills, our method enables smooth transitions to seamlessly compose multiple skills.