Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Human and AI Trust: Trust Attitude Measurement Instrument

Retno Larasati

TL;DR

This paper tackles the lack of a generalizable, validated instrument to measure trust in AI from a layperson perspective, specifically in healthcare AI. It develops a psychometrically grounded 16-item scale across eight trust domains through a six-stage process combining deductive theory with inductive data (surveys, focus groups, vignettes), expert validation, and cognitive interviews. The instrument demonstrates solid reliability (alpha/omega) and validity (content, convergent, some discriminant, concurrent) with robust CFA fit, though discriminant validity between some domains and predictive validity require further investigation. The work provides a practical, adaptable tool for researchers evaluating human-AI trust and offers a methodological blueprint for rigorous measurement in human-AI interaction research.

Abstract

With the current progress of Artificial Intelligence (AI) technology and its increasingly broader applications, trust is seen as a required criterion for AI usage, acceptance, and deployment. A robust measurement instrument is essential to correctly evaluate trust from a human-centered perspective. This paper describes the development and validation process of a trust measure instrument, which follows psychometric principles, and consists of a 16-items trust scale. The instrument was built explicitly for research in human-AI interaction to measure trust attitudes towards AI systems from layperson (non-expert) perspective. The use-case we used to develop the scale was in the context of AI medical support systems (specifically cancer/health prediction). The scale development (Measurement Item Development) and validation (Measurement Item Evaluation) involved six research stages: item development, item evaluation, survey administration, test of dimensionality, test of reliability, and test of validity. The results of the six-stages evaluation show that the proposed trust measurement instrument is empirically reliable and valid for systematically measuring and comparing non-experts' trust in AI Medical Support Systems.

Human and AI Trust: Trust Attitude Measurement Instrument

TL;DR

This paper tackles the lack of a generalizable, validated instrument to measure trust in AI from a layperson perspective, specifically in healthcare AI. It develops a psychometrically grounded 16-item scale across eight trust domains through a six-stage process combining deductive theory with inductive data (surveys, focus groups, vignettes), expert validation, and cognitive interviews. The instrument demonstrates solid reliability (alpha/omega) and validity (content, convergent, some discriminant, concurrent) with robust CFA fit, though discriminant validity between some domains and predictive validity require further investigation. The work provides a practical, adaptable tool for researchers evaluating human-AI trust and offers a methodological blueprint for rigorous measurement in human-AI interaction research.

Abstract

With the current progress of Artificial Intelligence (AI) technology and its increasingly broader applications, trust is seen as a required criterion for AI usage, acceptance, and deployment. A robust measurement instrument is essential to correctly evaluate trust from a human-centered perspective. This paper describes the development and validation process of a trust measure instrument, which follows psychometric principles, and consists of a 16-items trust scale. The instrument was built explicitly for research in human-AI interaction to measure trust attitudes towards AI systems from layperson (non-expert) perspective. The use-case we used to develop the scale was in the context of AI medical support systems (specifically cancer/health prediction). The scale development (Measurement Item Development) and validation (Measurement Item Evaluation) involved six research stages: item development, item evaluation, survey administration, test of dimensionality, test of reliability, and test of validity. The results of the six-stages evaluation show that the proposed trust measurement instrument is empirically reliable and valid for systematically measuring and comparing non-experts' trust in AI Medical Support Systems.
Paper Structure (39 sections, 1 equation, 1 figure, 16 tables)

This paper contains 39 sections, 1 equation, 1 figure, 16 tables.

Table of Contents

  1. Introduction
  2. Background and related work
  3. Trust Concepts
  4. Measuring Human Trust in AI Systems
  5. Measurement Instrument Development Process
  6. Item Development
  7. Methods
  8. Deductive Method
  9. Inductive Method
  10. Results: Trust Factors and Items Developed
  11. Deductive Method
  12. Inductive Method
  13. Item Evaluation
  14. Methods
  15. Expert Validation
  16. ...and 24 more sections

Figures (1)

  • Figure 1: Six Steps Process To Develop a Sound Human Trust Measurement Instrument.