Personality Perception in Human Videos Altered by Motion Transfer Networks

TL;DR

The study addresses how appearance and movement cues shape personality perception in videos altered by motion transfer networks. Using the Five-Factor model and Thin-Plate Spline Motion Model (TPS), it systematically examines how source versus driving inputs influence perceived traits across five factors. Two online user studies show motion cues strongly drive extraversion (and to a lesser extent openness) while appearance cues influence agreeableness and neuroticism, with conscientiousness being less affected. The findings illuminate how data-driven motion transfer can be leveraged to shape perceived personality in virtual characters, with practical implications for education, entertainment, and human-computer interaction.

Abstract

The successful portrayal of personality in digital characters improves communication and immersion. Current research focuses on expressing personality through modifying animations using heuristic rules or data-driven models. While studies suggest motion style highly influences the apparent personality, the role of appearance can be similarly essential. This work analyzes the influence of movement and appearance on the perceived personality of short videos altered by motion transfer networks. We label the personalities in conference video clips with a user study to determine the samples that best represent the Five-Factor model's high, neutral, and low traits. We alter these videos using the Thin-Plate Spline Motion Model, utilizing the selected samples as the source and driving inputs. We follow five different cases to study the influence of motion and appearance on personality perception. Our comparative study reveals that motion and appearance influence different factors: motion strongly affects perceived extraversion, and appearance helps convey agreeableness and neuroticism.

Figures (5)

• Figure 1: Selected samples representing high, low, and neutral traits for the corresponding personality factors. Each box plot summarizes the personality scores of the sample below; black lines depict the median, and the black dots show the mean.
• Figure 2: Example representative poses. Images on the right show the landmarks of the representative frames on the left.
• Figure 3: Sample motion transfer; the left column shows the representative frames from driving (top) and source (bottom) videos; the top right image shows the driving video and the bottom right image shows the transferred output.
• Figure 4: Box plots for the second study where participants' choice for Output A, Equal, and Output B are mapped to integers 0, 1, and 2, respectively. Black lines depict the median, and the black dots show the mean. Means close to 1 show that the corresponding combination does not help distinguish the high and low traits of the related personality. The first row examines the factors grouped by case, and the second row examines the cases grouped by personality.
• Figure 5: Consecutive frames of example outputs from TPS. Rows illustrate the frames that cover short segments of the outputs $A_{C2}$, $A_{C1}$, $A_{O2}$, $B_{A1}$, in order. The first output has artifacts where limb parts are mixed with the background, and the second output has artifacts in the background; the third output has a deformed face due to differences in facing directions. In contrast, the fourth output has no obvious artifacts.