MimicKit: A Reinforcement Learning Framework for Motion Imitation and Control

TL;DR

MimicKit addresses the challenge of learning motion controllers via reinforcement learning across diverse morphologies by providing a unified, modular framework built around an Agent–Model–Environment–Engine architecture. It unifies multiple imitation methods (DeepMimic, AMP, ASE, ADD) and employs PPO as the default algorithm, with AWR as an off-policy alternative, running on vectorized IsaacGym simulations for scalable data collection. The framework demonstrates trade-offs between tracking-based and distribution-matching approaches across humanoid and non-humanoid bodies, highlighting how reward shaping and termination criteria influence performance. Overall, MimicKit offers an open-source, reproducible platform to accelerate progress in learning-based motion control for graphics and robotics.

Abstract

MimicKit is an open-source framework for training motion controllers using motion imitation and reinforcement learning. The codebase provides implementations of commonly-used motion-imitation techniques and RL algorithms. This framework is intended to support research and applications in computer graphics and robotics by providing a unified training framework, along with standardized environment, agent, and data structures. The codebase is designed to be modular and easily configurable, enabling convenient modification and extension to new characters and tasks. The open-source codebase is available at: https://github.com/xbpeng/MimicKit.

Figures (8)

• Figure 1: Schematic overview of the MimicKit framework. The main components of the system are 1) the Agent, 2) the Model, 3) the Environment, and 4) the Engine. The learning algorithms are implemented primarily through the Agent and Model, while the Environment and Engine are responsible for simulating the desired task.
• Figure 2: Simulated Humanoid character.
• Figure 3: Snapshots of physically simulated characters performing skills learned by imitating motion data recorded from real-life actors. The methods implemented in MimicKit can be applied to train policies for a diverse cast of simulated characters and skills.
• Figure 4: Learning curves comparing the tracking performance with the simulated humanoid character trained with DeepMimic, AMP, and ADD. Five training runs initialized with different random seeds are shown for each method. In order to better compare methods under similar settings, policies are trained without pose-error termination. The standard configuration for tracking-based methods, such as DeepMimic and ADD, utilizes pose-error termination, which tends to produce better performance and more consistent results across training runs.
• Figure :