Always Tell Me The Odds: Fine-grained Conditional Probability Estimation

TL;DR

This work tackles fine-grained conditional probability estimation under uncertainty by introducing a decoder-based regression framework that outputs calibrated distributions for $P( ext{proposition} | ext{context})$. It combines synthetic data generation with reasoning-augmented prompts, an LLM-based judge, and rank-consistency training to supervise both regression and ranking objectives. By discretizing the target with $N$ bins and using an expected label scoring rule, the approach recovers precise probabilities while leveraging large back-end models and synthetic supervision to achieve strong performance across intrinsic, comparison, and structural tasks. The results demonstrate strong gains over fine-tuned encoders and prompting baselines, the benefit of synthetic data for domain generalization, and the model’s capacity to reflect human uncertainty in its probabilistic outputs, with practical implications for robust probabilistic reasoning in NLP systems.

Abstract

We present a state-of-the-art model for fine-grained probability estimation of propositions conditioned on context. Recent advances in large language models (LLMs) have significantly enhanced their reasoning capabilities, particularly on well-defined tasks with complete information. However, LLMs continue to struggle with making accurate and well-calibrated probabilistic predictions under uncertainty or partial information. While incorporating uncertainty into model predictions often boosts performance, obtaining reliable estimates of that uncertainty remains understudied. In particular, LLM probability estimates tend to be coarse and biased towards more frequent numbers. Through a combination of human and synthetic data creation and assessment, scaling to larger models, and better supervision, we propose a set of strong and precise probability estimation models. We conduct systematic evaluations across tasks that rely on conditional probability estimation and show that our approach consistently outperforms existing fine-tuned and prompting-based methods by a large margin.

Figures (8)

• Figure 1: We train decoder-based models for fine-grained probability estimation, going beyond human annotations. We rely on two sources of supervision: synthetic (left), and pairwise ranking (right). For synthetic data, we first collect multiple LLMs' probability estimates with reasoning, then group instances by LLM agreement. For those with significant discrepancies, we submit them to another LLM judge, which rate the quality of each reasoning process. These ratings are then used to aggregate the probability estimates into a distribution over possible bins. For pairwise ranking, we use a margin loss to ensure consistency between the pairwise labels and the fine-grained probability estimates -- which are computed via the expected label scoring rule.
• Figure 2: Illustration of our distribution quantization process. Notice that how the quantization preserves fine-grained label ordering and allows for better discrimination of targets that fall in the same bin.
• Figure 4: The Spearman Correlation over Pairwise Comparison Iterations.
• Figure 5: Analysis of GPT-4.0 on NLI probability pairwise comparison tests. (a) Higher label differences lead to improved accuracy and reduced entropy. (b) Frequency distribution of label differences across the sampled data.
• Figure : P: Ruth's 1927 single season record of 60 home runs stood unsurpassed until Roger Maris hit 61 in 1961. $\leadsto$H: Babe Ruth hit 60 home runs in his lifetime.