Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Perception of Emotions in Human and Robot Faces: Is the Eye Region Enough?

Chinmaya Mishra, Gabriel Skantze, Peter Hagoort, Rinus Verdonschot

TL;DR

This study investigates how robot face design—humanlike versus mechanical—and the use of full-face versus eye-region expressions influence emotion recognition in human–robot interaction. Using a between-subjects design (N=305) with two online experiments, the authors compare recognition accuracy for a human confederate, Hayden (humanlike) and Titan (mechanical) expressing six basic emotions plus Neutral, across full-face and eye-region videos, analyzed via a Generalized Linear Mixed Model (GLMM). Results show that fully animated, back-projected faces enable emotion recognition at levels comparable to humans, regardless of appearance, while restricting to the eye region reduces accuracy; within eye-region only, more human-like features enhance recognition. The findings inform design guidelines: model the whole face when possible, and if limited to the eye region, increase human-likeness to maintain emotion readability, with implications for practical social-robot design and HRI performance.

Abstract

The increased interest in developing next-gen social robots has raised questions about the factors affecting the perception of robot emotions. This study investigates the impact of robot appearances (humanlike, mechanical) and face regions (full-face, eye-region) on human perception of robot emotions. A between-subjects user study (N = 305) was conducted where participants were asked to identify the emotions being displayed in videos of robot faces, as well as a human baseline. Our findings reveal three important insights for effective social robot face design in Human-Robot Interaction (HRI): Firstly, robots equipped with a back-projected, fully animated face - regardless of whether they are more human-like or more mechanical-looking - demonstrate a capacity for emotional expression comparable to that of humans. Secondly, the recognition accuracy of emotional expressions in both humans and robots declines when only the eye region is visible. Lastly, within the constraint of only the eye region being visible, robots with more human-like features significantly enhance emotion recognition.

Perception of Emotions in Human and Robot Faces: Is the Eye Region Enough?

TL;DR

This study investigates how robot face design—humanlike versus mechanical—and the use of full-face versus eye-region expressions influence emotion recognition in human–robot interaction. Using a between-subjects design (N=305) with two online experiments, the authors compare recognition accuracy for a human confederate, Hayden (humanlike) and Titan (mechanical) expressing six basic emotions plus Neutral, across full-face and eye-region videos, analyzed via a Generalized Linear Mixed Model (GLMM). Results show that fully animated, back-projected faces enable emotion recognition at levels comparable to humans, regardless of appearance, while restricting to the eye region reduces accuracy; within eye-region only, more human-like features enhance recognition. The findings inform design guidelines: model the whole face when possible, and if limited to the eye region, increase human-likeness to maintain emotion readability, with implications for practical social-robot design and HRI performance.

Abstract

The increased interest in developing next-gen social robots has raised questions about the factors affecting the perception of robot emotions. This study investigates the impact of robot appearances (humanlike, mechanical) and face regions (full-face, eye-region) on human perception of robot emotions. A between-subjects user study (N = 305) was conducted where participants were asked to identify the emotions being displayed in videos of robot faces, as well as a human baseline. Our findings reveal three important insights for effective social robot face design in Human-Robot Interaction (HRI): Firstly, robots equipped with a back-projected, fully animated face - regardless of whether they are more human-like or more mechanical-looking - demonstrate a capacity for emotional expression comparable to that of humans. Secondly, the recognition accuracy of emotional expressions in both humans and robots declines when only the eye region is visible. Lastly, within the constraint of only the eye region being visible, robots with more human-like features significantly enhance emotion recognition.

Paper Structure

This paper contains 14 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Materials and Methods
  3. Robot Platform
  4. Robot Emotions
  5. Experimental Setup
  6. Participants and Procedure
  7. Results
  8. Effect of Appearances
  9. Effect of Facial Regions
  10. Discussion
  11. Conclusion
  12. Ethical Impact Statement
  13. Acknowledgments.
  14. Disclosure of Interests.

Figures (3)

  • Figure 1: Emotional expressions displayed by the three characters. The first row depicts the full-face expressions of Happy by the human confederate (A), the human-like robot character Hayden (B), and the mechanical-looking character Titan (C). The second row depicts the eye-region expressions for Sad by the human confederate (D), Hayden (E), and Titan (F)
  • Figure 2: Emotion recognition score for each of the three appearances and the combined results for both the face region conditions: full-face and eye-region. *** indicates a significant difference with $p < 0.001$
  • Figure 3: Normalized confusion matrix between actual and selected emotions by the participants under both the face region conditions. Sub-figure (A) depicts the confusion matrix for the full-face condition and (B) depicts the confusion matrix for the eye-region condition. Emotion abbreviation in the figure: H - Happy, Sa - Sad, F - Fear, A - Anger, Su - Surprise, D - Disgust, N - Neutral