Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Why Would You Suggest That? Human Trust in Language Model Responses

Manasi Sharma, Ho Chit Siu, Rohan Paleja, Jaime D. Peña

TL;DR

There is evidence that adding an explanation in the model response to justify its reasoning significantly increases self-reported user trust in the model when the user has the opportunity to compare various responses.

Abstract

The emergence of Large Language Models (LLMs) has revealed a growing need for human-AI collaboration, especially in creative decision-making scenarios where trust and reliance are paramount. Through human studies and model evaluations on the open-ended News Headline Generation task from the LaMP benchmark, we analyze how the framing and presence of explanations affect user trust and model performance. Overall, we provide evidence that adding an explanation in the model response to justify its reasoning significantly increases self-reported user trust in the model when the user has the opportunity to compare various responses. Position and faithfulness of these explanations are also important factors. However, these gains disappear when users are shown responses independently, suggesting that humans trust all model responses, including deceptive ones, equitably when they are shown in isolation. Our findings urge future research to delve deeper into the nuanced evaluation of trust in human-machine teaming systems.

Why Would You Suggest That? Human Trust in Language Model Responses

TL;DR

There is evidence that adding an explanation in the model response to justify its reasoning significantly increases self-reported user trust in the model when the user has the opportunity to compare various responses.

Abstract

The emergence of Large Language Models (LLMs) has revealed a growing need for human-AI collaboration, especially in creative decision-making scenarios where trust and reliance are paramount. Through human studies and model evaluations on the open-ended News Headline Generation task from the LaMP benchmark, we analyze how the framing and presence of explanations affect user trust and model performance. Overall, we provide evidence that adding an explanation in the model response to justify its reasoning significantly increases self-reported user trust in the model when the user has the opportunity to compare various responses. Position and faithfulness of these explanations are also important factors. However, these gains disappear when users are shown responses independently, suggesting that humans trust all model responses, including deceptive ones, equitably when they are shown in isolation. Our findings urge future research to delve deeper into the nuanced evaluation of trust in human-machine teaming systems.
Paper Structure (14 sections, 11 figures, 5 tables)

This paper contains 14 sections, 11 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Results
  5. Discussion
  6. Conclusion
  7. Acknowledgements
  8. Appendix
  9. Human-AI Interaction in Creative Settings
  10. Additional Related Works
  11. Methodology
  12. Additional LLM Results
  13. Participant Demographics
  14. Additional Limitations

Figures (11)

  • Figure 1: Combined heat maps and box plots of the three ranking groups. Significance bars omitted for clarity.
  • Figure 2: Combined heat maps and box plots of the three ranking groups. Significance bars omitted for clarity (see text).
  • Figure 3: ROUGE-1 and ROUGE-L mean and standard error scores for GPT-3.5 and GPT-4 results on LaMP-4 for the comparisons used in the ranking groups.
  • Figure 4: Bar Graph visualization of the results in Table \ref{['table:full_results_gpt3']} for LaMP-4, LaMP-5 and LaMP-7 for GPT-3.5. The random history baseline is visualized as a dotted line to compare other results to.
  • Figure 5: Bar Graph visualization of the results in Table \ref{['table:full_results_gpt4']} for LaMP-4, LaMP-5 and LaMP-7 for GPT-4. The random history baseline is visualized as a dotted line to compare other results to.
  • ...and 6 more figures