OpenEstimate: Evaluating LLMs on Reasoning Under Uncertainty with Real-World Data

TL;DR

OpenEstimate introduces an extensible benchmark to evaluate language models on probabilistic estimation under real-world uncertainty. By deriving 178 target variables from public tabular datasets and eliciting priors as Gaussian or Beta distributions, the framework assesses both accuracy and calibration against grounded ground-truth values. Across frontier LMs, priors are often overconfident and not consistently better than small-sample baselines, though certain elicitation strategies improve calibration and posterior usefulness when data are limited. The results highlight domain-dependent performance and establish a platform for developing models with improved probabilistic estimation and uncertainty reasoning in practical settings.

Abstract

Real-world settings where language models (LMs) are deployed -- in domains spanning healthcare, finance, and other forms of knowledge work -- require models to grapple with incomplete information and reason under uncertainty. Yet most LM evaluations focus on problems with well-defined answers and success criteria. This gap exists in part because natural problems involving uncertainty are difficult to construct: given that LMs have access to most of the same knowledge as humans, it is non-trivial to design questions for which LMs will struggle to produce correct answers, but which humans can answer reliably. As a result, LM performance on reasoning under uncertainty remains poorly characterized. To address this gap, we introduce OpenEstimate, an extensible, multi-domain benchmark for evaluating LMs on numerical estimation tasks that require models to synthesize significant amounts of background information and express predictions as probabilistic priors. We assess these priors for accuracy and calibration, quantifying their usefulness relative to samples from the true distribution of interest. Across six frontier LMs, we find that LM-elicited priors are often inaccurate and overconfident. Performance improves modestly depending on how uncertainty is elicited from the model, but is largely unaffected by changes in sampling strategy, reasoning effort, or prompt design. The OpenEstimate benchmark thus offers a challenging evaluation for frontier LMs and a platform for developing models that are better at probabilistic estimation and reasoning under uncertainty.

Figures (9)

• Figure 1: Variable generation and prior elicitation pipeline. We construct derived variables from large-scale observational datasets (e.g., PitchBook), specify them as statistical targets (e.g., Gaussian means), and prompt language models to provide Bayesian priors in the form of distributional parameters.
• Figure 2: MAE error ratio of LLM prior to a naive statistical baseline computed using a uninformative prior and five examples from the true distribution. Most models are no better than five examples; some are significantly worse. There isn't a statistically significant gap in performance between most model families.
• Figure 3: Expected calibration error (in percentage points) across domains and model families. The best model varies by domain, with reasoning models performing the best in Glassdoor and NHANES but not in Pitchbook. Again, most model families are not statistically different from each other in performance.
• Figure 4: Heatmap describing the deviations from perfect calibration of each approach. Bolded values are statistically significant according to a per-quartile binomial test ($p<0.05$). All approaches systematically overestimated across domains (Quartile 1 is greater than 25%). In some instances, there was high rates of both over and under-estimation (Quartile 1 and 4 are greater than 25%).
• Figure 5: Cumulative distribution function displaying the percentage of ground truth values that fall within $n\sigma$ standard deviations away from the mean of the prior, where $\sigma$ is the standard deviation of the prior. The dashed line represents perfect calibration for a Gaussian. The best performing models have 80% of the ground truths within 1-2.5 standard deviations from the prior mean.