PersonaAnimator: Personalized Motion Transfer from Unconstrained Videos

TL;DR

This work tackles the problem of creating identity-consistent digital avatars by introducing Video-to-Video Motion Personalization, a framework that learns personalized motion patterns directly from unconstrained videos. The authors present PersonaAnimator, featuring the Semantic-Aware Personalized Motion Transfer (SA-PMT) and Physics-aware Motion Style Regularization (PMSR) to decouple content motion from personalized style while enforcing physical plausibility. They also introduce PersonaVid, the first large-scale video dataset for personalized motion learning, spanning 20 content categories and 120 style categories under a one-person-one-style paradigm. Experimental results demonstrate state-of-the-art performance on PersonaVid and standard human animation datasets, with strong qualitative and generalization capabilities, enabling more expressive and authentic digital humans for applications in the metaverse, film, and games.

Abstract

Recent advances in motion generation show remarkable progress. However, several limitations remain: (1) Existing pose-guided character motion transfer methods merely replicate motion without learning its style characteristics, resulting in inexpressive characters. (2) Motion style transfer methods rely heavily on motion capture data, which is difficult to obtain. (3) Generated motions sometimes violate physical laws. To address these challenges, this paper pioneers a new task: Video-to-Video Motion Personalization. We propose a novel framework, PersonaAnimator, which learns personalized motion patterns directly from unconstrained videos. This enables personalized motion transfer. To support this task, we introduce PersonaVid, the first video-based personalized motion dataset. It contains 20 motion content categories and 120 motion style categories. We further propose a Physics-aware Motion Style Regularization mechanism to enforce physical plausibility in the generated motions. Extensive experiments show that PersonaAnimator outperforms state-of-the-art motion transfer methods and sets a new benchmark for the Video-to-Video Motion Personalization task.

Figures (5)

• Figure 1: Video-to-Video Motion Personalization. Our method preserves the basic motion content while learning personalized motion characteristics from the style motion video, achieving personalized transfer. Learned personalized features are marked with green bounding boxes.
• Figure 2: Architecture overview of PersonaAnimator framework. $\odot$ denotes the Hadamard product. $\oplus$ denotes the element-wise sum. $C$ denotes the concatenation operation.
• Figure 3: Examples of the PersonaVid dataset.
• Figure 4: Qualitative comparison. Our framework outperforms SOTA methods, with green/red boxes marking learned/unlearned motion features, respectively. The results demonstrate the superior performance of our framework.
• Figure 5: Generalization evaluation. We test different methods on an unseen style, with green/red boxes marking learned/unlearned motion features, respectively. Our method achieves better results than the SOTA methods.