Confidence in Large Language Model Evaluation: A Bayesian Approach to Limited-Sample Challenges

TL;DR

This work tackles the challenge of evaluating probabilistic LLM outputs under limited data by introducing a Bayesian framework that treats model capability as a latent trait $\theta$ and ranks a test model $\theta_x$ against anchor intervals $\theta_i<\theta_x\le\theta_{i+1}$. It formalizes ranking as Bayesian hypothesis testing over these intervals, uses a maximum-entropy prior to model $\theta$-space, and leverages a curated, discriminative query set to elicit informative responses. The approach integrates multi-trial observations through a likelihood that is averaged over boundary anchor models and combines them with priors to produce posterior interval memberships, enabling probabilistic statements about a model’s likelihood of surpassing baselines. Empirical results across GPT-series anchors and open-source test models show superior discriminative power and robustness to smaller query budgets compared to traditional accuracy, Pass@N, or mean$\pm$std metrics, with implications for adaptive anchor selection and practical deployment where data are scarce. Overall, the method advances LLM evaluation by uniting Bayesian inference with real-world constraints, offering calibrated, uncertainty-aware rankings that can guide model selection and policy decisions.

Abstract

Large language models (LLMs) exhibit probabilistic output characteristics, yet conventional evaluation frameworks rely on deterministic scalar metrics. This study introduces a Bayesian approach for LLM capability assessment that integrates prior knowledge through probabilistic inference, addressing limitations under limited-sample regimes. By treating model capabilities as latent variables and leveraging a curated query set to induce discriminative responses, we formalize model ranking as a Bayesian hypothesis testing problem over mutually exclusive capability intervals. Experimental evaluations with GPT-series models demonstrate that the proposed method achieves superior discrimination compared to conventional evaluation methods. Results indicate that even with reduced sample sizes, the approach maintains statistical robustness while providing actionable insights, such as probabilistic statements about a model's likelihood of surpassing specific baselines. This work advances LLM evaluation methodologies by bridging Bayesian inference with practical constraints in real-world deployment scenarios.

Figures (4)

• Figure 1: Anchor Model Performance(a) Success rates of the six anchor models (measured over $O=10$ trials per question) for $M=50$ evaluation questions. (b) The success rate of the first 20 questions are shown here, where the complete success rate distribution is provided in Appendix. Extreme probability values $\{0\%,100\%\}$ were modulated to $\{1\%, 99\%\}$ to ensure numerical stability during subsequent computations.
• Figure 2: Bayesian Probability Ranking AnalysisProbability distributions of the test model's ranking relative to six anchor models across varying question counts ($M$).The anchor models partitioned the ranking space into seven mutually exclusive intervals, with probabilities quantifying the likelihood of the test model falling into each interval.
• Figure 3: Method comparisonPerformance comparison between the proposed Bayesian approach and conventional evaluation metrics. Bayesian@1 and accuracy reporting are Single-trial evaluations, while Bayes@10, Pass@10 and Mean±std are aggregated from O=10 independent trials. All results are obtained from M=20 questions.
• Figure :