PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllers

TL;DR

PARC addresses the scarcity of motion data for agile terrain traversal by coupling a diffusion-based motion generator with a physics-based motion tracker in an iterative self-improvement loop. The motion generator creates synthetic terrain-traversal motions conditioned on local heightmaps, while the tracker corrects physics-based artifacts in simulation, and the corrected motions are fed back to further train the generator. Through multiple iterations, PARC expands the motion dataset, yielding novel, robust behaviors and a tracker capable of following increasingly complex trajectories, validated by quantitative metrics and qualitative demonstrations. This framework bridges the gap between limited high-quality motion data and the need for versatile, physically plausible controllers capable of navigating diverse terrains, with implications for animation, robotics, and interactive systems.

Abstract

Humans excel in navigating diverse, complex environments with agile motor skills, exemplified by parkour practitioners performing dynamic maneuvers, such as climbing up walls and jumping across gaps. Reproducing these agile movements with simulated characters remains challenging, in part due to the scarcity of motion capture data for agile terrain traversal behaviors and the high cost of acquiring such data. In this work, we introduce PARC (Physics-based Augmentation with Reinforcement Learning for Character Controllers), a framework that leverages machine learning and physics-based simulation to iteratively augment motion datasets and expand the capabilities of terrain traversal controllers. PARC begins by training a motion generator on a small dataset consisting of core terrain traversal skills. The motion generator is then used to produce synthetic data for traversing new terrains. However, these generated motions often exhibit artifacts, such as incorrect contacts or discontinuities. To correct these artifacts, we train a physics-based tracking controller to imitate the motions in simulation. The corrected motions are then added to the dataset, which is used to continue training the motion generator in the next iteration. PARC's iterative process jointly expands the capabilities of the motion generator and tracker, creating agile and versatile models for interacting with complex environments. PARC provides an effective approach to develop controllers for agile terrain traversal, which bridges the gap between the scarcity of motion data and the need for versatile character controllers.

Figures (20)

• Figure 1: Overview of the PARC framework. PARC iteratively trains a motion generator and motion tracker with self-generated motion data. The motion generator produces kinematic motion sequences to train the motion tracker, while the motion tracker corrects physics-related artifacts in a simulator, enabling the motion generator to continue training on new physics-based motions.
• Figure 2: The transformer encoder based architecture of the terrain-conditioned motion generator. ${\mathbf{h}}$ is first processed by a CNN into an image of shape 64x16x16, then unfolded into 64 non-overlapping image patches of shape 64x2x2. The image patches are then embedded into tokens with an MLP. The target direction ${\mathbf{d}}$ is embedded into a single token with an MLP. Each frame of the noisy motion sequence ${\mathbf{x}}_k$ is embedded into a token using an MLP.
• Figure 3: Long-horizon physics based motions generated using the final motion generator and motion tracker of PARC.
• Figure 4: Examples of terrain-traversal motions found in our original dataset. The terrain is typically very simple, and the vast majority of clips focus on showcasing one particular parkour skill such as jumping (top), running up stairs (middle), and climbing walls (bottom).
• Figure 5: A visualization of the distribution of the final relative horizontal (XY) root positions from motion clips in the dataset as at different PARC iterations. As the PARC iterations increase (left to right, top to bottom), the dataset expands and increases the diversity of trajectories.