High-Fidelity and Long-Duration Human Image Animation with Diffusion Transformer

TL;DR

This work tackles the challenge of producing long-duration, photorealistic human videos from a single reference image and driving motion. It introduces a diffusion-transformer framework that employs hybrid implicit guidance signals to enhance facial and hand fidelity, a Laplacian sharpness factor to preserve hand textures under motion blur, and a time-aware Position Shift Adaptive Module to support arbitrary video lengths. A data augmentation strategy and a skeleton-alignment model reduce identity-related shape variations, improving robustness across subjects. Comprehensive experiments show state-of-the-art performance on both quantitative metrics and human judgments, including the ability to generate videos longer than a minute. The approach holds strong potential for high-quality, long-form content in applications such as digital content creation and online communication.

Abstract

Recent progress in diffusion models has significantly advanced the field of human image animation. While existing methods can generate temporally consistent results for short or regular motions, significant challenges remain, particularly in generating long-duration videos. Furthermore, the synthesis of fine-grained facial and hand details remains under-explored, limiting the applicability of current approaches in real-world, high-quality applications. To address these limitations, we propose a diffusion transformer (DiT)-based framework which focuses on generating high-fidelity and long-duration human animation videos. First, we design a set of hybrid implicit guidance signals and a sharpness guidance factor, enabling our framework to additionally incorporate detailed facial and hand features as guidance. Next, we incorporate the time-aware position shift fusion module, modify the input format within the DiT backbone, and refer to this mechanism as the Position Shift Adaptive Module, which enables video generation of arbitrary length. Finally, we introduce a novel data augmentation strategy and a skeleton alignment model to reduce the impact of human shape variations across different identities. Experimental results demonstrate that our method outperforms existing state-of-the-art approaches, achieving superior performance in both high-fidelity and long-duration human image animation.

Figures (6)

• Figure 2: UniAnimate-DiT results using different long video generation strategies: (a) overlapped sliding window; (b) final frame extension; (c) final latent extension.
• Figure 3: Overview of the proposed method. Given a single reference image and a driving video, our framework generates high-fidelity, long-duration human motion videos with strong temporal coherence and detail preservation.
• Figure 4: Workflow of the Position Shift Adaptive Module. (a) shows the encoding process; (b) shows the decoding process.
• Figure 5: Qualitative comparison with other competing methods.
• Figure 6: Ablation on Hybrid Guidance Signals.