Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative Confidants: How do People Experience Trust in Emotional Support from Generative AI?

Riccardo Volpato, Simone Stumpf, Lisa DeBruine

TL;DR

This study investigates how people develop trust in emotional support from general-purpose generative AI using diary entries, chat transcripts, and in-depth interviews with 24 frequent users. The authors apply reflexive thematic analysis to identify drivers of trust, including personalisation, nuanced mental models of AI, and perceived user agency. They reveal a tension between the helpful, comforting aspects of AI language and concerns about accuracy, over-reliance, data privacy, and the overlap with formal therapy. The findings offer implications for safeguarding, AI design, and future research to support healthy, appropriate use of AI-based emotional support.

Abstract

People are increasingly turning to generative AI (e.g., ChatGPT, Gemini, Copilot) for emotional support and companionship. While trust is likely to play a central role in enabling these informal and unsupervised interactions, we still lack an understanding of how people develop and experience it in this context. Seeking to fill this gap, we recruited 24 frequent users of generative AI for emotional support and conducted a qualitative study consisting of diary entries about interactions, transcripts of chats with AI, and in-depth interviews. Our results suggest important novel drivers of trust in this context: familiarity emerging from personalisation, nuanced mental models of generative AI, and awareness of people's control over conversations. Notably, generative AI's homogeneous use of personalised, positive, and persuasive language appears to promote some of these trust-building factors. However, this also seems to discourage other trust-related behaviours, such as remembering that generative AI is a machine trained to converse in human language. We present implications for future research that are likely to become critical as the use of generative AI for emotional support increasingly overlaps with therapeutic work.

Generative Confidants: How do People Experience Trust in Emotional Support from Generative AI?

TL;DR

This study investigates how people develop trust in emotional support from general-purpose generative AI using diary entries, chat transcripts, and in-depth interviews with 24 frequent users. The authors apply reflexive thematic analysis to identify drivers of trust, including personalisation, nuanced mental models of AI, and perceived user agency. They reveal a tension between the helpful, comforting aspects of AI language and concerns about accuracy, over-reliance, data privacy, and the overlap with formal therapy. The findings offer implications for safeguarding, AI design, and future research to support healthy, appropriate use of AI-based emotional support.

Abstract

People are increasingly turning to generative AI (e.g., ChatGPT, Gemini, Copilot) for emotional support and companionship. While trust is likely to play a central role in enabling these informal and unsupervised interactions, we still lack an understanding of how people develop and experience it in this context. Seeking to fill this gap, we recruited 24 frequent users of generative AI for emotional support and conducted a qualitative study consisting of diary entries about interactions, transcripts of chats with AI, and in-depth interviews. Our results suggest important novel drivers of trust in this context: familiarity emerging from personalisation, nuanced mental models of generative AI, and awareness of people's control over conversations. Notably, generative AI's homogeneous use of personalised, positive, and persuasive language appears to promote some of these trust-building factors. However, this also seems to discourage other trust-related behaviours, such as remembering that generative AI is a machine trained to converse in human language. We present implications for future research that are likely to become critical as the use of generative AI for emotional support increasingly overlaps with therapeutic work.
Paper Structure (73 sections, 4 figures, 1 table)

This paper contains 73 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Emotional Support in the Digital Era
  4. Emotional Support from Generative AI
  5. Trust in Generative AI
  6. Methods
  7. Participants
  8. Procedure and Data Collection
  9. Data Analysis
  10. Diaries and Interviews
  11. Chat Transcripts
  12. Reporting
  13. Limitations
  14. Results: Diaries and Interviews
  15. Motivations
  16. ...and 58 more sections

Figures (4)

  • Figure 1: Themes from diary entries and interviews. Numbers in brackets show how many of the 24 participants each theme applies to across diaries and interviews.
  • Figure 2: Stages in the development of trust in generative AI for emotional support.
  • Figure 3: Chat transcript statistics. (A): Mean message length per participant, comparing AI and user contributions. (B): K-means clusters of participants based on average number of messages and average message length. The legend at the top shows cluster names and colours across the two charts.
  • Figure 4: Themes from chat transcripts. Numbers in brackets indicate the participants whose transcripts the theme appears in.