UniCon: Universal Neural Controller For Physics-based Character Motion

TL;DR

UniCon proposes a universal neural controller for physics-based character motion by separating control into a robust low-level executor and interchangeable high-level schedulers. Through constrained multi-objective reward optimization, motion balancer, and adaptive variance, the low-level policy efficiently learns to reproduce thousands of motions with strong generalization and zero-shot robustness. The framework supports real-time interactive inputs (keyboard, video, motion stitching) without retraining, demonstrating improved efficiency, robustness, and transferability over prior methods. This approach significantly advances scalable, interactive, physics-based animation with broad applicability to games and film.

Abstract

The field of physics-based animation is gaining importance due to the increasing demand for realism in video games and films, and has recently seen wide adoption of data-driven techniques, such as deep reinforcement learning (RL), which learn control from (human) demonstrations. While RL has shown impressive results at reproducing individual motions and interactive locomotion, existing methods are limited in their ability to generalize to new motions and their ability to compose a complex motion sequence interactively. In this paper, we propose a physics-based universal neural controller (UniCon) that learns to master thousands of motions with different styles by learning on large-scale motion datasets. UniCon is a two-level framework that consists of a high-level motion scheduler and an RL-powered low-level motion executor, which is our key innovation. By systematically analyzing existing multi-motion RL frameworks, we introduce a novel objective function and training techniques which make a significant leap in performance. Once trained, our motion executor can be combined with different high-level schedulers without the need for retraining, enabling a variety of real-time interactive applications. We show that UniCon can support keyboard-driven control, compose motion sequences drawn from a large pool of locomotion and acrobatics skills and teleport a person captured on video to a physics-based virtual avatar. Numerical and qualitative results demonstrate a significant improvement in efficiency, robustness and generalizability of UniCon over prior state-of-the-art, showcasing transferability to unseen motions, unseen humanoid models and unseen perturbation.

Figures (14)

• Figure 1: Overview of UniCon. Our model consists of ( right) an RL-powered low-level motion executor that is able to physically animate a given (non-physically plausible) sequence of target motion frames. The low-level motion executor can work in conjunction with a plethora of ( left) high-level motion schedulers which produce target motion frames.
• Figure 2: The figures here visualize the training process, where the white characters are driven by ground-truth motions, and the green ones are trained characters. We use the motion dataset training scheduler to train our low-level motion executor. 4096 characters are used to generate training samples.
• Figure 3: Diagram of interactive control from video.
• Figure 4: Diagram of keyboard driven interactive control scheduler.
• Figure 5: Diagram of motion stitching scheduler.