DreamActor-M2: Universal Character Image Animation via Spatiotemporal In-Context Learning

TL;DR

DreamActor-M2 reframes motion conditioning as a spatiotemporal in-context learning problem to unify appearance and motion cues for universal character animation. It introduces a two-stage pipeline (pose-based then end-to-end RGB-driven) and a self-bootstrapped data synthesis strategy to achieve robust generalization across humans, animals, cartoons, and multi-person scenes. A new AWBench benchmark enables thorough cross-domain evaluation, where DreamActor-M2 achieves state-of-the-art visual fidelity and cross-domain generalization, validated by both automatic metrics and human judgments. The work promises scalable, pose-free animation across diverse subjects, with practical implications for entertainment, VFX, and digital media, while acknowledging ethical considerations and the need for responsible use.

Abstract

Character image animation aims to synthesize high-fidelity videos by transferring motion from a driving sequence to a static reference image. Despite recent advancements, existing methods suffer from two fundamental challenges: (1) suboptimal motion injection strategies that lead to a trade-off between identity preservation and motion consistency, manifesting as a "see-saw", and (2) an over-reliance on explicit pose priors (e.g., skeletons), which inadequately capture intricate dynamics and hinder generalization to arbitrary, non-humanoid characters. To address these challenges, we present DreamActor-M2, a universal animation framework that reimagines motion conditioning as an in-context learning problem. Our approach follows a two-stage paradigm. First, we bridge the input modality gap by fusing reference appearance and motion cues into a unified latent space, enabling the model to jointly reason about spatial identity and temporal dynamics by leveraging the generative prior of foundational models. Second, we introduce a self-bootstrapped data synthesis pipeline that curates pseudo cross-identity training pairs, facilitating a seamless transition from pose-dependent control to direct, end-to-end RGB-driven animation. This strategy significantly enhances generalization across diverse characters and motion scenarios. To facilitate comprehensive evaluation, we further introduce AW Bench, a versatile benchmark encompassing a wide spectrum of characters types and motion scenarios. Extensive experiments demonstrate that DreamActor-M2 achieves state-of-the-art performance, delivering superior visual fidelity and robust cross-domain generalization. Project Page: https://grisoon.github.io/DreamActor-M2/

Figures (10)

• Figure 1: The proposed DreamActor-M2 exhibits strong generalization capability, producing diverse animations while preserving consistent character appearance. The arrow $\xrightarrow{}$ denotes the transfer of character motion from the driving video to the character depicted in the reference image.
• Figure 2: The schematic overview of proposed DreamActor-M2.
• Figure 3: Visual examples of driving video corpus and reference image corpus.
• Figure 4: Qualitative comparisons between our method and state-of-the-art approaches on AWBench.
• Figure 5: GSB comparison of DreamActor-M2 and other products.