Kling-MotionControl Technical Report

TL;DR

Kling-MotionControl is presented, a unified DiT-based framework engineered specifically for robust, precise, and expressive holistic character animation, achieving exceptional fidelity in holistic motion control, open domain generalization, and visual quality and coherence.

Abstract

Character animation aims to generate lifelike videos by transferring motion dynamics from a driving video to a reference image. Recent strides in generative models have paved the way for high-fidelity character animation. In this work, we present Kling-MotionControl, a unified DiT-based framework engineered specifically for robust, precise, and expressive holistic character animation. Leveraging a divide-and-conquer strategy within a cohesive system, the model orchestrates heterogeneous motion representations tailored to the distinct characteristics of body, face, and hands, effectively reconciling large-scale structural stability with fine-grained articulatory expressiveness. To ensure robust cross-identity generalization, we incorporate adaptive identity-agnostic learning, facilitating natural motion retargeting for diverse characters ranging from realistic humans to stylized cartoons. Simultaneously, we guarantee faithful appearance preservation through meticulous identity injection and fusion designs, further supported by a subject library mechanism that leverages comprehensive reference contexts. To ensure practical utility, we implement an advanced acceleration framework utilizing multi-stage distillation, boosting inference speed by over 10x. Kling-MotionControl distinguishes itself through intelligent semantic motion understanding and precise text responsiveness, allowing for flexible control beyond visual inputs. Human preference evaluations demonstrate that Kling-MotionControl delivers superior performance compared to leading commercial and open-source solutions, achieving exceptional fidelity in holistic motion control, open domain generalization, and visual quality and coherence. These results establish Kling-MotionControl as a robust solution for high-quality, controllable, and lifelike character animation.

Figures (5)

• Figure 1: Given a reference image and a driving video, Kling-MotionControl generates high-fidelity videos where the reference subject faithfully mimics multi-granular driving motions, encompassing body movements, facial expressions, and hand gestures. Our results demonstrate precise fine-grained control, robustness against rapid and complex dynamics, and natural cross-identity transfer with faithful identity preservation, generalizing seamlessly to diverse characters such as anime, cartoons, and stylized artworks.
• Figure 2: Overview of training and inference pipeline of our proposed Kling-MotionControl.
• Figure 3: Visualization of GSB evaluation results (preference rates in percentages) comparing Kling-MotionControl with Dreamina, Runway Act-Two, and Wan-Animate across various evaluation dimensions. Note that numerical labels are omitted for categories with 0%.
• Figure 4: Qualitative comparisons with baseline methods. Kling-MotionControl delivers high-fidelity holistic character animation videos, characterized by exceptional expressiveness and motion accuracy, while maintaining faithful identity and scene consistency. Top: Our method produces more vivid and precise facial expressions and hand gestures. Bottom: Our framework exhibits superior robustness against complex and rapid body motion, yielding higher-fidelity results.
• Figure 5: Visualization of generated results across diverse scenarios. We highlight Kling-MotionControl's capability to generate high-fidelity character animations with accurate motion imitation ranging from complex body dynamics to fine-grained facial and gestural details, while faithfully preserving appearance across various identities and maintaining excellent text controllability. The leftmost column displays reference images, with generated results and driving sequences (insets) in the remaining columns.