Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Can Pose Transfer Models Generate Realistic Human Motion?

Vaclav Knapp, Matyas Bohacek

TL;DR

This study evaluates three state-of-the-art video pose transfer methods—AnimateAnyone, MagicAnimate, and ExAvatar—on out-of-domain identities and actions using a human-subject survey. By constructing a dataset with 60 reference actions from UCF101 and novel identities (image-based for diffusion methods; video-based for ExAvatar), the authors generate 840 videos per method and select 22 for evaluation. The results show limited ability to convey correct actions (overall 42.92% accuracy) and varying consistency with source videos (36.46% overall), with ExAvatar performing best (68.12% action accuracy and 55.00% consistency) but still leaving substantial room for improvement (38.54% inconsistent). The findings highlight challenges in photorealism and semantic fidelity under OOD conditions and argue for hybrid 3D-graphics plus AI approaches and the development of stronger generalization benchmarks that incorporate human evaluations. Collectively, the work pinpoints practical barriers to real-world deployment of pose transfer and guides future research toward more robust, semantically faithful, and photorealistic video generation.

Abstract

Recent pose-transfer methods aim to generate temporally consistent and fully controllable videos of human action where the motion from a reference video is reenacted by a new identity. We evaluate three state-of-the-art pose-transfer methods -- AnimateAnyone, MagicAnimate, and ExAvatar -- by generating videos with actions and identities outside the training distribution and conducting a participant study about the quality of these videos. In a controlled environment of 20 distinct human actions, we find that participants, presented with the pose-transferred videos, correctly identify the desired action only 42.92% of the time. Moreover, the participants find the actions in the generated videos consistent with the reference (source) videos only 36.46% of the time. These results vary by method: participants find the splatting-based ExAvatar more consistent and photorealistic than the diffusion-based AnimateAnyone and MagicAnimate.

Can Pose Transfer Models Generate Realistic Human Motion?

TL;DR

This study evaluates three state-of-the-art video pose transfer methods—AnimateAnyone, MagicAnimate, and ExAvatar—on out-of-domain identities and actions using a human-subject survey. By constructing a dataset with 60 reference actions from UCF101 and novel identities (image-based for diffusion methods; video-based for ExAvatar), the authors generate 840 videos per method and select 22 for evaluation. The results show limited ability to convey correct actions (overall 42.92% accuracy) and varying consistency with source videos (36.46% overall), with ExAvatar performing best (68.12% action accuracy and 55.00% consistency) but still leaving substantial room for improvement (38.54% inconsistent). The findings highlight challenges in photorealism and semantic fidelity under OOD conditions and argue for hybrid 3D-graphics plus AI approaches and the development of stronger generalization benchmarks that incorporate human evaluations. Collectively, the work pinpoints practical barriers to real-world deployment of pose transfer and guides future research toward more robust, semantically faithful, and photorealistic video generation.

Abstract

Recent pose-transfer methods aim to generate temporally consistent and fully controllable videos of human action where the motion from a reference video is reenacted by a new identity. We evaluate three state-of-the-art pose-transfer methods -- AnimateAnyone, MagicAnimate, and ExAvatar -- by generating videos with actions and identities outside the training distribution and conducting a participant study about the quality of these videos. In a controlled environment of 20 distinct human actions, we find that participants, presented with the pose-transferred videos, correctly identify the desired action only 42.92% of the time. Moreover, the participants find the actions in the generated videos consistent with the reference (source) videos only 36.46% of the time. These results vary by method: participants find the splatting-based ExAvatar more consistent and photorealistic than the diffusion-based AnimateAnyone and MagicAnimate.
Paper Structure (17 sections, 8 figures, 4 tables)

This paper contains 17 sections, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Image-level Pose Transfer
  4. Video-level Pose Transfer
  5. Out-of-Domain Evaluation
  6. Data and Methods
  7. Reference Action Videos
  8. Novel Human Identities
  9. Distribution Analysis
  10. Pose Transfer
  11. Survey Methodology
  12. Results
  13. Discussion
  14. Conclusion
  15. Selected Action Classes
  16. ...and 2 more sections

Figures (8)

  • Figure 1: Representative examples of video frames generated by AnimateAnyone, MagicAnimate, and ExAvatar. Shown on the left are frames where the generated human motion is consistent with the reference action video (source) and appears photorealistic; on the right are frames where the generated human motion is not consistent with the reference action video (source) or does not appear photorealistic.
  • Figure 2: Representative examples of frames from UCF101, cropped to fit the layout.
  • Figure 3: Representative examples of video frames from the RANDOM People dataset, cropped to fit the layout.
  • Figure 4: Representative examples of target identities.
  • Figure 5: t-SNE visualization of CLIP embeddings representing datasets used in the OOD evaluation (UCF101 and RANDOM People) and datasets used to train the evaluated pose transfer methods (TikTok Dataset and X-Humans).
  • ...and 3 more figures