Trust Me on This: A User Study of Trustworthiness for RAG Responses

TL;DR

The paper investigates how explanations influence user trust in retrieval-augmented generation (RAG) systems, addressing transparency gaps when sources and limitations are hidden in synthesized answers. It employs a two-stage, within-subject user study that compares pairs of responses differing in objective quality and then exposes the same pairs with one of three explanation types: source attribution, factual grounding, or information coverage. Results show that explanations guide users toward higher-quality answers, but trust does not always align with objective quality; factors such as clarity, actionability, and the user's prior knowledge substantially shape judgments, with grounding having the strongest effect in factual contexts. The work suggests designing adaptive, query-type–aware explanations and personalizing explanations to user knowledge, contributing to more calibrated trust in RAG-based information systems.

Abstract

The integration of generative AI into information access systems often presents users with synthesized answers that lack transparency. This study investigates how different types of explanations can influence user trust in responses from retrieval-augmented generation systems. We conducted a controlled, two-stage user study where participants chose the more trustworthy response from a pair-one objectively higher quality than the other-both with and without one of three explanation types: (1) source attribution, (2) factual grounding, and (3) information coverage. Our results show that while explanations significantly guide users toward selecting higher quality responses, trust is not dictated by objective quality alone: Users' judgments are also heavily influenced by response clarity, actionability, and their own prior knowledge.

Figures (2)

• Figure 1: User study design (example on a single query).
• Figure 2: Proportion of user trust judgments for each explanation type, comparing responses with and without explanations. Colors indicate the direction of preference.