Character Mixing for Video Generation

TL;DR

This work tackles inter-character interaction in text-to-video generation by addressing non-coexistence and style delusion with Cross-Character Embedding (CCE) and Cross-Character Augmentation (CCA). CCE learns character identity and behavior through character–action prompting, enabling coherent cross-domain interactions, while CCA introduces synthetic cross-domain data using segmentation and compositing to preserve style fidelity. A curated 81-hour, multi-domain dataset and a new benchmark enable quantitative evaluation of identity, motion, interaction, and style across single- and multi-subject scenarios, with Gemini-based metrics showing improvements over state-of-the-art baselines. The approach enables controllable multi-character video synthesis across diverse universes, advancing storytelling capabilities, though it requires explicit identity annotations and targeted fine-tuning for new characters.

Abstract

Imagine Mr. Bean stepping into Tom and Jerry--can we generate videos where characters interact naturally across different worlds? We study inter-character interaction in text-to-video generation, where the key challenge is to preserve each character's identity and behaviors while enabling coherent cross-context interaction. This is difficult because characters may never have coexisted and because mixing styles often causes style delusion, where realistic characters appear cartoonish or vice versa. We introduce a framework that tackles these issues with Cross-Character Embedding (CCE), which learns identity and behavioral logic across multimodal sources, and Cross-Character Augmentation (CCA), which enriches training with synthetic co-existence and mixed-style data. Together, these techniques allow natural interactions between previously uncoexistent characters without losing stylistic fidelity. Experiments on a curated benchmark of cartoons and live-action series with 10 characters show clear improvements in identity preservation, interaction quality, and robustness to style delusion, enabling new forms of generative storytelling.Additional results and videos are available on our project page: https://tingtingliao.github.io/mimix/.

Figures (14)

• Figure 1: Multi-character Mixing. Our method preserves character identity, behavior and original style while generating plausible interactions between characters that have never coexisted—from cartoons (We Bare Bears, Tom and Jerry) to realistic humans (Mr. Bean, Young Sheldon).
• Figure 2: Style delusion examples. When mixing different style characters, their styles may shift undesirably. For instance, Mr. Bean looks cartoonish (top row), while Ice Bear appears realistic (bottom row).
• Figure 3: Model finetuning architecture (left) and data augmentation pipeline (right).
• Figure 4: Comparison on multi-subject interaction. Results from SkyReel-A2 fei2025skyreelsa2 (top row) and ours (bottom row).
• Figure 5: Comparison on single-subject generation. From top to bottom: results from VideoBooth videobooth, DreamVideo, Wan2.1-I2V wan2025video, SkyReel-A2 fei2025skyreelsa2 and ours.