"I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust

TL;DR

This work investigates how natural-language uncertainty expressions in LLM outputs influence user reliance, trust, and performance during medical information seeking. Through a large-scale, preregistered MTurk experiment with four conditions, the authors show that first-person uncertainty expressions reduce user confidence and agreement with AI while increasing accuracy, suggesting reduced overreliance; general uncertainty yields weaker effects. The findings highlight that uncertainty signaling can calibrate user behavior but may trade off with trust, and that language choices require careful user testing before deployment. The study informs design and policy discussions around uncertainty communication in LLM-infused systems to mitigate overreliance without eroding adoptive trust.

Abstract

Widely deployed large language models (LLMs) can produce convincing yet incorrect outputs, potentially misleading users who may rely on them as if they were correct. To reduce such overreliance, there have been calls for LLMs to communicate their uncertainty to end users. However, there has been little empirical work examining how users perceive and act upon LLMs' expressions of uncertainty. We explore this question through a large-scale, pre-registered, human-subject experiment (N=404) in which participants answer medical questions with or without access to responses from a fictional LLM-infused search engine. Using both behavioral and self-reported measures, we examine how different natural language expressions of uncertainty impact participants' reliance, trust, and overall task performance. We find that first-person expressions (e.g., "I'm not sure, but...") decrease participants' confidence in the system and tendency to agree with the system's answers, while increasing participants' accuracy. An exploratory analysis suggests that this increase can be attributed to reduced (but not fully eliminated) overreliance on incorrect answers. While we observe similar effects for uncertainty expressed from a general perspective (e.g., "It's not clear, but..."), these effects are weaker and not statistically significant. Our findings suggest that using natural language expressions of uncertainty may be an effective approach for reducing overreliance on LLMs, but that the precise language used matters. This highlights the importance of user testing before deploying LLMs at scale.

Figures (4)

• Figure 1: Screenshot of the information-seeking task(shown in the Control, Uncertain1st, and UncertainGeneral conditions). Additional screenshots, including the variant shown in the No-AI condition, can be found in Appendix D.
• Figure 2: Exploratory analysis of over- and underreliance (\ref{['sec:separateanalysis']}).We analyze Correct (%) separately for questions the system answered correctly vs. incorrectly. We show the model-estimated means and standard errors for each condition (\ref{['fig:appropriatereliance_h1']}) and for not uncertain vs. uncertain responses in the conditions with uncertainty (\ref{['fig:appropriatereliance_h2']}).
• Figure D3: Task example page shown for participants in the Control, Uncertain1st, UncertainGeneral conditions.
• Figure D4: Study stimuli for one of the tasks.The question is the same, but participants in different conditions were provided with different AI responses (e.g., with vs. without uncertainty) or not provided with an AI response.