Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploratory Models of Human-AI Teams: Leveraging Human Digital Twins to Investigate Trust Development

Daniel Nguyen, Myke C. Cohen, Hsien-Te Kao, Grant Engberson, Louis Penafiel, Spencer Lynch, Svitlana Volkova

TL;DR

This paper addresses three research questions relating to the use of digital twins for modeling trust in HATs, and discusses the properties of HDT trust: self-report measures, interaction-based measures, and compliance type behavioral measures.

Abstract

As human-agent teaming (HAT) research continues to grow, computational methods for modeling HAT behaviors and measuring HAT effectiveness also continue to develop. One rising method involves the use of human digital twins (HDT) to approximate human behaviors and socio-emotional-cognitive reactions to AI-driven agent team members. In this paper, we address three research questions relating to the use of digital twins for modeling trust in HATs. First, to address the question of how we can appropriately model and operationalize HAT trust through HDT HAT experiments, we conducted causal analytics of team communication data to understand the impact of empathy, socio-cognitive, and emotional constructs on trust formation. Additionally, we reflect on the current state of the HAT trust science to discuss characteristics of HAT trust that must be replicable by a HDT such as individual differences in trust tendencies, emergent trust patterns, and appropriate measurement of these characteristics over time. Second, to address the question of how valid measures of HDT trust are for approximating human trust in HATs, we discuss the properties of HDT trust: self-report measures, interaction-based measures, and compliance type behavioral measures. Additionally, we share results of preliminary simulations comparing different LLM models for generating HDT communications and analyze their ability to replicate human-like trust dynamics. Third, to address how HAT experimental manipulations will extend to human digital twin studies, we share experimental design focusing on propensity to trust for HDTs vs. transparency and competency-based trust for AI agents.

Exploratory Models of Human-AI Teams: Leveraging Human Digital Twins to Investigate Trust Development

TL;DR

This paper addresses three research questions relating to the use of digital twins for modeling trust in HATs, and discusses the properties of HDT trust: self-report measures, interaction-based measures, and compliance type behavioral measures.

Abstract

As human-agent teaming (HAT) research continues to grow, computational methods for modeling HAT behaviors and measuring HAT effectiveness also continue to develop. One rising method involves the use of human digital twins (HDT) to approximate human behaviors and socio-emotional-cognitive reactions to AI-driven agent team members. In this paper, we address three research questions relating to the use of digital twins for modeling trust in HATs. First, to address the question of how we can appropriately model and operationalize HAT trust through HDT HAT experiments, we conducted causal analytics of team communication data to understand the impact of empathy, socio-cognitive, and emotional constructs on trust formation. Additionally, we reflect on the current state of the HAT trust science to discuss characteristics of HAT trust that must be replicable by a HDT such as individual differences in trust tendencies, emergent trust patterns, and appropriate measurement of these characteristics over time. Second, to address the question of how valid measures of HDT trust are for approximating human trust in HATs, we discuss the properties of HDT trust: self-report measures, interaction-based measures, and compliance type behavioral measures. Additionally, we share results of preliminary simulations comparing different LLM models for generating HDT communications and analyze their ability to replicate human-like trust dynamics. Third, to address how HAT experimental manipulations will extend to human digital twin studies, we share experimental design focusing on propensity to trust for HDTs vs. transparency and competency-based trust for AI agents.

Paper Structure

This paper contains 29 sections, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Research Question 1: How can we model and measure HAT trust dynamics in HDTs?
  3. Theoretical Considerations for Modeling HAT Trust in HDTATs
  4. Evolution of Trust Definitions in Human-AI Contexts
  5. Trust Formation and Reliance in Human-AI Teams
  6. Beyond Capability-Based Trust
  7. Trust as a Dynamic Social Construct
  8. Methods for Measuring HAT Trust
  9. Self-reported Trust
  10. Behavioral Trust
  11. Emerging Techniques for Measuring Trust
  12. Empirical Investigations into HAT Trust using Exploratory Causal Analysis
  13. Causal Analysis Results: How Empathy Constructs Effect HAT Trust?
  14. Causal Analysis Results: How Socio-cognitive Constructs Effect HAT Trust
  15. Causal Analysis Results: How Emotion Constructs Effect HAT Trust
  16. ...and 14 more sections

Figures (6)

  • Figure 1: Results of causal analysis on how empathy constructs effect HAT trust.
  • Figure 2: Results of exploratory causal analysis of socio-cognitive indicators on trust indicators.
  • Figure 3: Results of exploratory causal analysis of emotion indicators on trust indicators.
  • Figure 4: Deviations of simulated conversations from the ground truth (orange) across model and level of prompt with respect to conversational dominance (left) and entropy (right).
  • Figure 5: Deviations of simulated conversations from the ground truth (orange) across model and level of prompt with respect to negative (left) and positive (right) sentiment
  • ...and 1 more figures