Morphology-Independent Facial Expression Imitation for Human-Face Robots

TL;DR

A morphology-independent expression imitation method that decouples expressions from facial morphology to eliminate morphological influence and produce more realistic expressions for human-face robots is proposed.

Abstract

Accurate facial expression imitation on human-face robots is crucial for achieving natural human-robot interaction. Most existing methods have achieved photorealistic expression imitation through mapping 2D facial landmarks to a robot's actuator commands. Their imitation of landmark trajectories is susceptible to interference from facial morphology, which would lead to a performance drop. In this paper, we propose a morphology-independent expression imitation method that decouples expressions from facial morphology to eliminate morphological influence and produce more realistic expressions for human-face robots. Specifically, we construct an expression decoupling module to learn expression semantics by disentangling the expression representation from the morphology representation in a self-supervised manner. We devise an expression transfer module to map the representations to the robot's actuator commands through a learning objective of perceiving expression errors, producing accurate facial expressions based on the learned expression semantics. To support experimental validation, a custom-designed and highly expressive human-face robot, namely Pengrui, is developed to serve as an experimental platform for realistic expression imitation. Extensive experiments demonstrate that our method enables the human-face robot to reproduce a wide range of human-like expressions effectively. All code and implementation details of the robot will be released.

Figures (6)

• Figure 1: The overview of the morphology-independent facial expression imitation method. An facial image is processed by the expression decoupling module to extract the disentanglemented representations of facial expressions, morphology and poses. The expression representations are then mapped to actuator commands via the expression transfer module, actuating Pengrui, i.e., our custom-designed and highl expressive human-face robot, to reproduce the target expression.
• Figure 2: Illustration of the expression decoupling module and the expression transfer module.
• Figure 3: Pengrui, a highly articulated human-face robot platform. Pengrui robot has $32$ actuators with $48$ degrees of freedom.
• Figure 4: Representation Visualizations. t-SNE visualization comparing the morphology-independent representation (our method) and landmark-based representation.
• Figure 5: Examples of interferences from facial morphology. The 2D facial landmarks for the (a) happy expression on a narrow face, (b) happy expression on a broad face and (c) angry expression on a broad face, are mapped to low-dimensional space using the t-SNE algorithm. The distance for the same expression across different morphology shown in (a) and (b) significantly exceeds that for different expressions on the same morphology shown in (a) and (c).