From Evidence to Belief: A Bayesian Epistemology Approach to Language Models

TL;DR

This work probes whether language models possess knowledge by framing belief as probabilistic confidence within Bayesian epistemology. It builds a dataset of evidence with varying informativeness and reliability and evaluates LLMs using verbalized confidence, token probability, and sampling across Confirmation and Strength-of-Evidence tasks. Key findings show partial alignment with Bayesian normative expectations: models reliably increase confidence with true gold evidence (Confirmation), but fail to consistently disconfirm or ignore irrelevant evidence, and exhibit a bias toward gold-standard information. The results illuminate why LLMs deviate from Bayesian updating, highlight calibration challenges, and suggest directions to improve reliability and interpretability in evidence-based answering. Overall, the study provides a philosophical and empirical lens on belief updating in LLMs with implications for calibration, evaluation, and the design of robust AI systems.

Abstract

This paper investigates the knowledge of language models from the perspective of Bayesian epistemology. We explore how language models adjust their confidence and responses when presented with evidence with varying levels of informativeness and reliability. To study these properties, we create a dataset with various types of evidence and analyze language models' responses and confidence using verbalized confidence, token probability, and sampling. We observed that language models do not consistently follow Bayesian epistemology: language models follow the Bayesian confirmation assumption well with true evidence but fail to adhere to other Bayesian assumptions when encountering different evidence types. Also, we demonstrated that language models can exhibit high confidence when given strong evidence, but this does not always guarantee high accuracy. Our analysis also reveals that language models are biased toward golden evidence and show varying performance depending on the degree of irrelevance, helping explain why they deviate from Bayesian assumptions.

Figures (6)

• Figure 1: The overall experimental method and simple examples of the evidence dataset for Confirmation Task. As golden evidence that aligns with the question is given to language models, it shows high confidence and accuracy. However, if language models encounter irrelevant evidence, it responds with low confidence. Further results and analysis are reported in Section \ref{['subsec:confirm']}.
• Figure 2: The results of the Strength of Evidence task on the SciQ dataset with verbal confidence method. The blue bar indicates more credible, specific, recent, and experimental evidence, while the red bar represents less credible, less specific, older, and observational evidence provided to the LLMs. We found that, in all models and datasets, strong evidence leads to high confidence with verbalized confidence. However, it does not always result in improvements in ACC and ECE.
• Figure 3: The results for the degree of variations in evidence for the SciQ dataset with verbalized method. We modified the number of conflicting sentences in conflicting evidence, sentences in incomplete evidence, and contradictory sentences in contradictory evidence (See Appendix \ref{['apdx:golden']} for entire results).
• Figure 4: The results of the Strength of Evidence task on the SciQ dataset with token probability method. The blue bar represents the cases where the strength of evidence is high. Specifically, the blue bar indicates the context from more credible sources, more specific, recent, and experimental evidence, while the red color represents less credible sources, less specific, old, and observational evidence.
• Figure 5: The results of the Strength of Evidence task on the SciQ dataset with sampling method. The blue bar represents the cases where the strength of evidence is high. Specifically, the blue bar indicates the context from more credible sources, more specific, recent, and experimental evidence, while the red color represents less credible sources, less specific, old, and observational evidence.