Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SIMBA UQ: Similarity-Based Aggregation for Uncertainty Quantification in Large Language Models

Debarun Bhattacharjya, Balaji Ganesan, Junkyu Lee, Radu Marinescu, Katsiaryna Mirylenka, Michael Glass, Xiao Shou

TL;DR

This work tackles uncertainty quantification for large language models by reframing confidence as a function of similarity among multiple generated samples. It introduces SIMBA, a unifying framework that uses sampling, pairwise similarities, and a similarity-driven aggregation to estimate per-sample confidences in a largely black-box setting. The paper presents three aggregation strategies—Simple, Bayesian, and Aggregation by Classification (random forest)—and demonstrates through extensive experiments on QA, summarization, and text-to-SQL tasks that similarity-based confidences can achieve good calibration (ACE) and competitive discrimination (ATS, AUROC). While gains are sometimes moderate and rely on in-domain training data for the supervised methods, the results suggest significant practical potential for trusted AI systems that require calibrated, non-verbalized UQ across diverse generative tasks.

Abstract

When does a large language model (LLM) know what it does not know? Uncertainty quantification (UQ) provides measures of uncertainty, such as an estimate of the confidence in an LLM's generated output, and is therefore increasingly recognized as a crucial component of trusted AI systems. Black-box UQ methods do not require access to internal model information from the generating LLM and therefore have numerous real-world advantages, such as robustness to system changes, adaptability to choice of LLM, reduced costs, and computational tractability. In this paper, we investigate the effectiveness of UQ techniques that are primarily but not necessarily entirely black-box, where the consistency between a generated output and other sampled generations is used as a proxy for confidence in its correctness. We propose a high-level non-verbalized similarity-based aggregation framework that subsumes a broad swath of UQ approaches suitable for complex generative tasks, as well as introduce specific novel techniques from the framework that train confidence estimation models using small training sets. Through an empirical study with datasets spanning the diverse tasks of question answering, summarization, and text-to-SQL, we demonstrate that our proposed similarity-based methods can yield better calibrated confidences than baselines.

SIMBA UQ: Similarity-Based Aggregation for Uncertainty Quantification in Large Language Models

TL;DR

This work tackles uncertainty quantification for large language models by reframing confidence as a function of similarity among multiple generated samples. It introduces SIMBA, a unifying framework that uses sampling, pairwise similarities, and a similarity-driven aggregation to estimate per-sample confidences in a largely black-box setting. The paper presents three aggregation strategies—Simple, Bayesian, and Aggregation by Classification (random forest)—and demonstrates through extensive experiments on QA, summarization, and text-to-SQL tasks that similarity-based confidences can achieve good calibration (ACE) and competitive discrimination (ATS, AUROC). While gains are sometimes moderate and rely on in-domain training data for the supervised methods, the results suggest significant practical potential for trusted AI systems that require calibrated, non-verbalized UQ across diverse generative tasks.

Abstract

When does a large language model (LLM) know what it does not know? Uncertainty quantification (UQ) provides measures of uncertainty, such as an estimate of the confidence in an LLM's generated output, and is therefore increasingly recognized as a crucial component of trusted AI systems. Black-box UQ methods do not require access to internal model information from the generating LLM and therefore have numerous real-world advantages, such as robustness to system changes, adaptability to choice of LLM, reduced costs, and computational tractability. In this paper, we investigate the effectiveness of UQ techniques that are primarily but not necessarily entirely black-box, where the consistency between a generated output and other sampled generations is used as a proxy for confidence in its correctness. We propose a high-level non-verbalized similarity-based aggregation framework that subsumes a broad swath of UQ approaches suitable for complex generative tasks, as well as introduce specific novel techniques from the framework that train confidence estimation models using small training sets. Through an empirical study with datasets spanning the diverse tasks of question answering, summarization, and text-to-SQL, we demonstrate that our proposed similarity-based methods can yield better calibrated confidences than baselines.
Paper Structure (25 sections, 1 equation, 3 figures, 5 tables)

This paper contains 25 sections, 1 equation, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. White-Box Methods.
  4. Black-Box Methods.
  5. Comparison with Prior Work.
  6. Methodology
  7. Basic Notation and Assumptions
  8. Sampling
  9. Computing Pairwise Similarities
  10. Similarity Aggregation
  11. Empirical Investigation
  12. Experimental Setup
  13. Datasets.
  14. Models.
  15. Evaluation Metrics.
  16. ...and 10 more sections

Figures (3)

  • Figure 1: Framework for UQ yielding confidence estimates for generations from an LLM in response to a natural language query. An illustrative natural language query from the Spider dataset for the text-to-SQL task is shown, along with example outputs at each step of the pipeline. Jaccard is chosen as the similarity metric, and confidences are estimated using a proposed 'aggregation by classification' method with random forests (described later).
  • Figure 2: Effect of choice of similarity metric on the ACE metric for the CoQA, SamSum, and Spider datasets.
  • Figure 3: Three different ablations on the ACE metric for the CoQA dataset.