Micro and macro facial expressions by driven animations in realistic Virtual Humans

TL;DR

This study investigates how macro and micro facial expressions are perceived in realistic virtual humans when expressions are transferred from real actors via facial tracking (driven animation) versus created by artists using modeling (modeled animation). Using a realistic VH built in MetaHuman Creator and Live Link Face, the authors transfer expressions from the DISFA+ (macro) and SAMM (micro) datasets and compare perception along recognition, intensity, and comfort, with a secondary test distinguishing real from driven expressions. OpenFace AU analysis reveals that most transferred expressions differ in intensity from real ones, indicating smoothing effects in driven animation. The results show that driven animations yield lower emotion-recognition accuracy than modeled animations, though participants can still identify the corresponding real expressions at high overall rates, suggesting driven approaches can be industry-viable with caveats. Overall, the work provides empirical guidance for when to favor artist-driven modeling versus real-actor driven animation in realistic VHs and highlights the trade-offs between realism, perceptual accuracy, and production scalability.

Abstract

Computer Graphics (CG) advancements have allowed the creation of more realistic Virtual Humans (VH) through modern techniques for animating the VH body and face, thereby affecting perception. From traditional methods, including blend shapes, to driven animations using facial and body tracking, these advancements can potentially enhance the perception of comfort and realism in relation to VHs. Previously, Psychology studied facial movements in humans, with some works separating expressions into macro and micro expressions. Also, some previous CG studies have analyzed how macro and micro expressions are perceived, replicating psychology studies in VHs, encompassing studies with realistic and cartoon VHs, and exploring different VH technologies. However, instead of using facial tracking animation methods, these previous studies animated the VHs using blendshapes interpolation. To understand how the facial tracking technique alters the perception of VHs, this paper extends the study to macro and micro expressions, employing two datasets to transfer real facial expressions to VHs and analyze how their expressions are perceived. Our findings suggest that transferring facial expressions from real actors to VHs significantly diminishes the accuracy of emotion perception compared to VH facial animations created by artists.

Figures (4)

• Figure 1: All macro expressions represented by the realistic character used in our work.
• Figure 2: All micro expressions represented by the realistic character used in our work.
• Figure 3: Figure of the first part of the survey. Firstly, it is presented a CG video, presenting one of the six basic emotions, presented in Figures \ref{['fig:macro-expressions']} and \ref{['fig:micro-expressions']}, followed by the three questions outlined in Table \ref{['tab:questions-p1']}.
• Figure 4: Figure of the second part of the survey. It starts with three videos, followed by a question. One of the three videos features a real human, while the other two are VHs. For macro expressions, both videos express the same emotion but are transferred from different actors, one of them the same as the reference video. For micro expressions, one video is the animation transferred from the reference video, while the other is a neutral face. The reference video used in this example is from DISFA+ dataset mavadati2016extended.