Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The ATTUNE model for Artificial Trust Towards Human Operators

Giannis Petousakis, Angelo Cangelosi, Rustam Stolkin, Manolis Chiou

TL;DR

The paper addresses quantifying AI agent trust toward a human operator in task-specific HRI by proposing ATTUNE, a framework that fuses Theory of Mind-derived context (human state, intent, actions) with performance signals to estimate trust in real time. ATTUNE blends attention-based, navigational-intent, and performance confidences through a conditional weighted average, augmented by short-term and long-term memory modules and a Trust Coefficient Factor to reflect incident-driven adjustments. The approach is validated on ROSbag recordings from a disaster-site teleoperation dataset, showing that the AI trust ranking tracks operator performance and qualitative assessments while revealing limitations and avenues for expansion. This work advances human-robot teaming by enabling robots to monitor and reason about human partners with a computed trust level, potentially guiding adaptive autonomy and supervision in high-stakes tasks.

Abstract

This paper presents a novel method to quantify Trust in HRI. It proposes an HRI framework for estimating the Robot Trust towards the Human in the context of a narrow and specified task. The framework produces a real-time estimation of an AI agent's Artificial Trust towards a Human partner interacting with a mobile teleoperation robot. The approach for the framework is based on principles drawn from Theory of Mind, including information about the human state, action, and intent. The framework creates the ATTUNE model for Artificial Trust Towards Human Operators. The model uses metrics on the operator's state of attention, navigational intent, actions, and performance to quantify the Trust towards them. The model is tested on a pre-existing dataset that includes recordings (ROSbags) of a human trial in a simulated disaster response scenario. The performance of ATTUNE is evaluated through a qualitative and quantitative analysis. The results of the analyses provide insight into the next stages of the research and help refine the proposed approach.

The ATTUNE model for Artificial Trust Towards Human Operators

TL;DR

The paper addresses quantifying AI agent trust toward a human operator in task-specific HRI by proposing ATTUNE, a framework that fuses Theory of Mind-derived context (human state, intent, actions) with performance signals to estimate trust in real time. ATTUNE blends attention-based, navigational-intent, and performance confidences through a conditional weighted average, augmented by short-term and long-term memory modules and a Trust Coefficient Factor to reflect incident-driven adjustments. The approach is validated on ROSbag recordings from a disaster-site teleoperation dataset, showing that the AI trust ranking tracks operator performance and qualitative assessments while revealing limitations and avenues for expansion. This work advances human-robot teaming by enabling robots to monitor and reason about human partners with a computed trust level, potentially guiding adaptive autonomy and supervision in high-stakes tasks.

Abstract

This paper presents a novel method to quantify Trust in HRI. It proposes an HRI framework for estimating the Robot Trust towards the Human in the context of a narrow and specified task. The framework produces a real-time estimation of an AI agent's Artificial Trust towards a Human partner interacting with a mobile teleoperation robot. The approach for the framework is based on principles drawn from Theory of Mind, including information about the human state, action, and intent. The framework creates the ATTUNE model for Artificial Trust Towards Human Operators. The model uses metrics on the operator's state of attention, navigational intent, actions, and performance to quantify the Trust towards them. The model is tested on a pre-existing dataset that includes recordings (ROSbags) of a human trial in a simulated disaster response scenario. The performance of ATTUNE is evaluated through a qualitative and quantitative analysis. The results of the analyses provide insight into the next stages of the research and help refine the proposed approach.

Paper Structure

This paper contains 20 sections, 6 equations, 4 figures.

Table of Contents

  1. INTRODUCTION
  2. Background and Related Work
  3. Trust
  4. Theory of Mind
  5. ToM and intent recognition
  6. ToM, and Trust in HRI
  7. Triadic Human-AI Agent-Operator Interaction
  8. Related Work
  9. Artificial Trust Framework and the ATTUNE model
  10. Artificial Trust Framework
  11. ATTUNE Model
  12. Artificial Trust Metric
  13. Attention-based Confidence
  14. Navigational Intent-based Confidence
  15. Performance-based Confidence
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Diagram of the ATTUNE Model
  • Figure 2: Ranking of the Operators based on the AI agent Trust
  • Figure 3: Comparison of the best and worst operator's estimated trust
  • Figure 4: Operator's capability metrics sorted for Trust