Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoning

TL;DR

The paper introduces the Martingale Score, a regression-based, unsupervised metric for detecting belief entrenchment and deviations from Bayesian rationality in LLM reasoning. By analyzing belief updates across iterations in open-ended tasks, it demonstrates that entrenchment is widespread across models, domains, and prompts and is associated with declines in ground-truth accuracy where labels exist. The work validates the measure through cross-model judge consistency and human alignment, and shows its applicability to open-ended domains beyond easily verifiable ground-truth tasks. This process-based metric offers a practical tool for assessing truth-seeking behavior in LLMs and lays the groundwork for reducing entrenchment in future research and applications.

Abstract

Recent advances in reasoning techniques have substantially improved the performance of large language models (LLMs), raising expectations for their ability to provide accurate, truthful, and reliable information. However, emerging evidence suggests that iterative reasoning may foster belief entrenchment and confirmation bias, rather than enhancing truth-seeking behavior. In this study, we propose a systematic evaluation framework for belief entrenchment in LLM reasoning by leveraging the Martingale property from Bayesian statistics. This property implies that, under rational belief updating, the expected value of future beliefs should remain equal to the current belief, i.e., belief updates are unpredictable from the current belief. We propose the unsupervised, regression-based Martingale Score to measure violations of this property, which signal deviation from the Bayesian ability of updating on new evidence. In open-ended problem domains including event forecasting, value-laden questions, and academic paper review, we find such violations to be widespread across models and setups, where the current belief positively predicts future belief updates, a phenomenon which we term belief entrenchment. We identify the models, reasoning techniques, and domains more prone to belief entrenchment. Finally, we validate the Martingale Score by showing that it predicts ground-truth accuracy on problem domains where ground truth labels are available. This indicates that, while designed as an unsupervised metric that operates even in domains without access to ground truth, the Martingale Score is a useful proxy of the truth-seeking ability of a reasoning process.

Figures (7)

• Figure 1: Example of Belief Entrenchment: LLM progressively updates beliefs in favor of its prior belief. Its belief update is highly predictable from the prior, violating the Martingale property.
• Figure 2: An illustration of Martingale Score calculation in our setting. "Prior Belief" refers to the expressed beliefs in most immediate LLM output; whereas "Posterior Belief" usually refers to the terminal beliefs after extended reasoning or engagements with external evidence. "Prior" and "Posterior" are relative concepts and their difference is taken as "Belief Update". We estimate the linear coefficient when running a linear regression between belief updates and prior beliefs. The positive value of the linear coefficient is our practical choice of the Martingale Score, measuring the predictability of belief update solely based on prior belief.
• Figure 3: All experiment setups. (a) Example reasoning trajectory. (b) CoT reasoning and debate reasoning, where in the latter, one model is instructed to debate with its clone. (c) Problem domains: forecasting questions from Metaculus and Polymarket; value-laden questions from r/ChangeMyView, where owners of posts share statements they hold strong beliefs in, expecting counterarguments; acceptance decisions for ICLR submissions given the abstract and reviews. (d) Models evaluated. (e) System prompts used. "Prior-conforming prompt" instructs the model to fixate on their prior beliefs, whereas "critical thinking" prompt encourages models to challenge their prior beliefs. The two prompts represent the extreme behaviors we intend to demonstrate.
• Figure 4: Relationship between the absolute value of the Martingale Score and the Brier Score. The former indicates the predictability of belief update based solely on the prior belief, and the latter measures the accuracy of probabilistic predictions. We observed that the Martingale Score and Brier score are positively correlated across all setups, suggesting belief entrenchment harms accuracy on binary problems. Taking "CoT on Forecasting" as an example, a Martingale Score of $0.0$ corresponds to a Brier score smaller than 0.25, slightly better than random guess ($0.250$) halawi2024approaching; in contrast, when the model is mildly entrenched on its prior belief (marked by Martingale Score of $0.04$), the forecasting performance is worse than random guess.
• Figure 5: Increased absolute value of the Martingale Score is associated with worse prediction accuracy (higher Brier Scores) and explains a significant portion of the latter's variance. In each regression model, we predict the Brier Score with the absolute value of the Martingale Score, while controlling for different potential confounders, including problem domain, reasoning techniques ("RM"), choice of model, and choice of prompt.

Theorems & Definitions (1)

• Proposition 1