Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Using Physiological Measures, Gaze, and Facial Expressions to Model Human Trust in a Robot Partner

Haley N. Green, Tariq Iqbal

TL;DR

This work addresses the problem of real-time, objective prediction of human trust in a robot partner by leveraging a multimodal, non-invasive sensor suite (EDA, BVP, TEMP, gaze, and facial AUs) collected in a supervisory HRI task. A novel in-person dataset was created where participants supervised a Panda robot performing a sandwich assembly task under varying success conditions, with a secondary task to modulate cognitive load; trust labels were obtained via the Muir scale and aligned to task epochs. Six classifiers (RF, ET, LDA, LR, DT, SVM) were trained and evaluated using 10-fold cross-validation, revealing that Extra Trees, Random Forest, and Decision Trees consistently outperform others, with the best configuration (EDA+BVP+TEMP+GAZE) achieving 97.5% accuracy. The results demonstrate that combining physiological signals and gaze can yield a robust, real-time trust model for human-robot collaboration, enabling safer and more productive interactions, and point to potential personalization via baseline normalization. The study also provides an open path for deploying lightweight trust predictors on robots and highlights avenues for future work in real-world settings and finer-grained trust scales.

Abstract

With robots becoming increasingly prevalent in various domains, it has become crucial to equip them with tools to achieve greater fluency in interactions with humans. One of the promising areas for further exploration lies in human trust. A real-time, objective model of human trust could be used to maximize productivity, preserve safety, and mitigate failure. In this work, we attempt to use physiological measures, gaze, and facial expressions to model human trust in a robot partner. We are the first to design an in-person, human-robot supervisory interaction study to create a dedicated trust dataset. Using this dataset, we train machine learning algorithms to identify the objective measures that are most indicative of trust in a robot partner, advancing trust prediction in human-robot interactions. Our findings indicate that a combination of sensor modalities (blood volume pulse, electrodermal activity, skin temperature, and gaze) can enhance the accuracy of detecting human trust in a robot partner. Furthermore, the Extra Trees, Random Forest, and Decision Trees classifiers exhibit consistently better performance in measuring the person's trust in the robot partner. These results lay the groundwork for constructing a real-time trust model for human-robot interaction, which could foster more efficient interactions between humans and robots.

Using Physiological Measures, Gaze, and Facial Expressions to Model Human Trust in a Robot Partner

TL;DR

This work addresses the problem of real-time, objective prediction of human trust in a robot partner by leveraging a multimodal, non-invasive sensor suite (EDA, BVP, TEMP, gaze, and facial AUs) collected in a supervisory HRI task. A novel in-person dataset was created where participants supervised a Panda robot performing a sandwich assembly task under varying success conditions, with a secondary task to modulate cognitive load; trust labels were obtained via the Muir scale and aligned to task epochs. Six classifiers (RF, ET, LDA, LR, DT, SVM) were trained and evaluated using 10-fold cross-validation, revealing that Extra Trees, Random Forest, and Decision Trees consistently outperform others, with the best configuration (EDA+BVP+TEMP+GAZE) achieving 97.5% accuracy. The results demonstrate that combining physiological signals and gaze can yield a robust, real-time trust model for human-robot collaboration, enabling safer and more productive interactions, and point to potential personalization via baseline normalization. The study also provides an open path for deploying lightweight trust predictors on robots and highlights avenues for future work in real-world settings and finer-grained trust scales.

Abstract

With robots becoming increasingly prevalent in various domains, it has become crucial to equip them with tools to achieve greater fluency in interactions with humans. One of the promising areas for further exploration lies in human trust. A real-time, objective model of human trust could be used to maximize productivity, preserve safety, and mitigate failure. In this work, we attempt to use physiological measures, gaze, and facial expressions to model human trust in a robot partner. We are the first to design an in-person, human-robot supervisory interaction study to create a dedicated trust dataset. Using this dataset, we train machine learning algorithms to identify the objective measures that are most indicative of trust in a robot partner, advancing trust prediction in human-robot interactions. Our findings indicate that a combination of sensor modalities (blood volume pulse, electrodermal activity, skin temperature, and gaze) can enhance the accuracy of detecting human trust in a robot partner. Furthermore, the Extra Trees, Random Forest, and Decision Trees classifiers exhibit consistently better performance in measuring the person's trust in the robot partner. These results lay the groundwork for constructing a real-time trust model for human-robot interaction, which could foster more efficient interactions between humans and robots.

Paper Structure

This paper contains 36 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Trust in Human-Human Interactions
  4. Trust in Human-Robot Interactions
  5. Trust and Physiological Response
  6. Method
  7. Experimental Scenario
  8. Primary Task - Sandwich Assembly
  9. Secondary Task - Word Searches
  10. The Robot
  11. Participants
  12. Measurement
  13. Objective Measures
  14. Physiological Measures
  15. Gaze
  16. ...and 21 more sections

Figures (2)

  • Figure 1: Sample view of a participant supervising a robot (left) and completing their word search task (right). The Empatica E4 watch is worn on the non-dominant hand with EDA extender leads attached to the index and middle fingers, highlighted on the left.
  • Figure 2: Sample view of the successful robot's performance (left) and the failing robot's performance (right). In the failure case, the robot stores the final ingredient instead of stacking it in the sandwich.