Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Teaching Language Models to Faithfully Express their Uncertainty

Bryan Eikema, Evgenia Ilia, José G. C. de Souza, Chrysoula Zerva, Wilker Aziz

TL;DR

This work tackles the problem that large language models often express uncertainty in ways that do not faithfully reflect their internal beliefs. It introduces Faithful Uncertainty Tuning (FUT), a lightweight fine-tuning approach that teaches instruction-tuned LLMs to hedge their answers in a manner that aligns with the model's own confidence without altering the underlying content distribution. By generating a specialized training dataset from unbiased model samples and hedging based on sample-consistency, FUT preserves beliefs while producing faithful uncertainty expressions; evaluation on open-domain QA across multiple models and datasets shows substantial gains in faithfulness (as measured by $cMFG$) with minimal semantic distribution shift and little QA loss. The method is robust to hedge choices, decoding strategies, and even numeric expressions of uncertainty, offering a practical pathway to more trustworthy AI interactions. These findings suggest FUT as a simple, generalizable tool to improve user trust and interpretability in AI systems without expensive re-training for correctness.

Abstract

Large language models (LLMs) often miscommunicate their uncertainty: repeated queries can produce divergent answers, yet generated responses are typically unhedged or hedged in ways that do not reflect this variability. This conveys unfaithful information about the uncertain state of the LLMs' knowledge, creating a faithfulness gap that affects even strong LLMs. We introduce Faithful Uncertainty Tuning (FUT): a fine-tuning approach that teaches instruction-tuned LLMs to express uncertainty faithfully without altering their underlying answer distribution. We construct training data by augmenting model samples with uncertainty hedges (i.e. verbal cues such as 'possibly' or 'likely') aligned with sample consistency, requiring no supervision beyond the model and a set of prompts. We evaluate FUT on open-domain question answering (QA) across multiple models and datasets. Our results show that FUT substantially reduces the faithfulness gap, while preserving QA accuracy and introducing minimal semantic distribution shift. Further analyses demonstrate robustness across decoding strategies, choice of hedgers, and other forms of uncertainty expression (i.e. numerical). These findings establish FUT as a simple and effective way to teach LLMs to communicate uncertainty faithfully.

Teaching Language Models to Faithfully Express their Uncertainty

TL;DR

This work tackles the problem that large language models often express uncertainty in ways that do not faithfully reflect their internal beliefs. It introduces Faithful Uncertainty Tuning (FUT), a lightweight fine-tuning approach that teaches instruction-tuned LLMs to hedge their answers in a manner that aligns with the model's own confidence without altering the underlying content distribution. By generating a specialized training dataset from unbiased model samples and hedging based on sample-consistency, FUT preserves beliefs while producing faithful uncertainty expressions; evaluation on open-domain QA across multiple models and datasets shows substantial gains in faithfulness (as measured by ) with minimal semantic distribution shift and little QA loss. The method is robust to hedge choices, decoding strategies, and even numeric expressions of uncertainty, offering a practical pathway to more trustworthy AI interactions. These findings suggest FUT as a simple, generalizable tool to improve user trust and interpretability in AI systems without expensive re-training for correctness.

Abstract

Large language models (LLMs) often miscommunicate their uncertainty: repeated queries can produce divergent answers, yet generated responses are typically unhedged or hedged in ways that do not reflect this variability. This conveys unfaithful information about the uncertain state of the LLMs' knowledge, creating a faithfulness gap that affects even strong LLMs. We introduce Faithful Uncertainty Tuning (FUT): a fine-tuning approach that teaches instruction-tuned LLMs to express uncertainty faithfully without altering their underlying answer distribution. We construct training data by augmenting model samples with uncertainty hedges (i.e. verbal cues such as 'possibly' or 'likely') aligned with sample consistency, requiring no supervision beyond the model and a set of prompts. We evaluate FUT on open-domain question answering (QA) across multiple models and datasets. Our results show that FUT substantially reduces the faithfulness gap, while preserving QA accuracy and introducing minimal semantic distribution shift. Further analyses demonstrate robustness across decoding strategies, choice of hedgers, and other forms of uncertainty expression (i.e. numerical). These findings establish FUT as a simple and effective way to teach LLMs to communicate uncertainty faithfully.
Paper Structure (42 sections, 2 equations, 6 figures, 12 tables)

This paper contains 42 sections, 2 equations, 6 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Measuring Faithfulness
  3. Faithful Uncertainty Tuning (FUT)
  4. Faithful response generation.
  5. Hedging.
  6. Belief preservation.
  7. Training.
  8. Experiments
  9. Faithfulness Evaluation
  10. Faithfulness Judge.
  11. Faithfulness Results.
  12. Out-of-Domain Performance.
  13. Example Generations.
  14. Preserving Model Beliefs
  15. Impact on QA Performance.
  16. ...and 27 more sections

Figures (6)

  • Figure 1: We want to know Who pushed Big Bird? and, for transparency’s sake, we expect a good response to be suggestive of uncertainty in the model’s state of knowledge. Each plot shows the beliefs of a model expressed as probabilities over clustered responses, with sampled responses shown below the plot. We segment the vertical axes in 5 intervals, and highlight hedge phrases that humans regard as coherent with those intervals of probabilistic belief. The original model (left) generates responses that are either unhedged or hedged unfaithfully (e.g. the model’s belief in Elmo is in the ballpark of 'possible’, but responses blaming Elmo suggest 'certainty’), leading to misinformation. Our method (right) adapts the model so that responses are faithfully hedged while closely preserving the original beliefs.
  • Figure 2: Total variation distance (TVD) between distributions over semantically distinct response clusters for OLMo2 13B. Shown are TVD values for different model–prompt comparisons (left) and pairwise differences of these values (right).
  • Figure 3: Distribution of faithful generation scores (left) for OLMo2 13B variants: the base model (vanilla prompt, Base), and FUT (interweave, postfix, and numerical). We also plot the instance-level differences from the Base model (right).
  • Figure 4: Scatter plots of confidence vs. decisiveness for failure cases: instances with reduced faithful generation values after fine-tuning (top) and those below 0.5 (bottom). Colours show changes relative to the base OLMo2 13B (vanilla prompt), with negative values indicating drops and vice versa.
  • Figure 5: An exact reproduction of Figure 5 from vogel2022interpretation: averaged mean values (with 95% confidence intervals) for 35 verbal probability expressions aggregated across prior studies.
  • ...and 1 more figures