Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Thermometer: Towards Universal Calibration for Large Language Models

Maohao Shen, Subhro Das, Kristjan Greenewald, Prasanna Sattigeri, Gregory Wornell, Soumya Ghosh

TL;DR

THERMOMETER tackles the universal calibration of large language models by learning a shared recognition network that predicts dataset-specific temperatures τ across multiple tasks. The approach uses a variational lower bound to jointly model per-task temperatures with amortized inference, enabling test-time estimation from unlabeled data and preserving the uncalibrated model's accuracy. Empirical results across MMLU, BIG-bench, and MRQA show consistent calibration improvements with minimal runtime overhead and strong transfer across model scales and benchmarks. The method demonstrates practical impact for deploying calibrated LLMs in diverse, real-world settings where labeled data for calibration is scarce or unavailable.

Abstract

We consider the issue of calibration in large language models (LLM). Recent studies have found that common interventions such as instruction tuning often result in poorly calibrated LLMs. Although calibration is well-explored in traditional applications, calibrating LLMs is uniquely challenging. These challenges stem as much from the severe computational requirements of LLMs as from their versatility, which allows them to be applied to diverse tasks. Addressing these challenges, we propose THERMOMETER, a calibration approach tailored to LLMs. THERMOMETER learns an auxiliary model, given data from multiple tasks, for calibrating a LLM. It is computationally efficient, preserves the accuracy of the LLM, and produces better-calibrated responses for new tasks. Extensive empirical evaluations across various benchmarks demonstrate the effectiveness of the proposed method.

Thermometer: Towards Universal Calibration for Large Language Models

TL;DR

THERMOMETER tackles the universal calibration of large language models by learning a shared recognition network that predicts dataset-specific temperatures τ across multiple tasks. The approach uses a variational lower bound to jointly model per-task temperatures with amortized inference, enabling test-time estimation from unlabeled data and preserving the uncalibrated model's accuracy. Empirical results across MMLU, BIG-bench, and MRQA show consistent calibration improvements with minimal runtime overhead and strong transfer across model scales and benchmarks. The method demonstrates practical impact for deploying calibrated LLMs in diverse, real-world settings where labeled data for calibration is scarce or unavailable.

Abstract

We consider the issue of calibration in large language models (LLM). Recent studies have found that common interventions such as instruction tuning often result in poorly calibrated LLMs. Although calibration is well-explored in traditional applications, calibrating LLMs is uniquely challenging. These challenges stem as much from the severe computational requirements of LLMs as from their versatility, which allows them to be applied to diverse tasks. Addressing these challenges, we propose THERMOMETER, a calibration approach tailored to LLMs. THERMOMETER learns an auxiliary model, given data from multiple tasks, for calibrating a LLM. It is computationally efficient, preserves the accuracy of the LLM, and produces better-calibrated responses for new tasks. Extensive empirical evaluations across various benchmarks demonstrate the effectiveness of the proposed method.
Paper Structure (39 sections, 1 theorem, 16 equations, 16 figures, 16 tables, 1 algorithm)

This paper contains 39 sections, 1 theorem, 16 equations, 16 figures, 16 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. $\textsc{thermometer}$
  5. A Variational Lower-bound
  6. Test Time Procedure
  7. Question Answering with Free Form Answers
  8. Experiments
  9. Main Results
  10. Multiple-choice QA
  11. QA with Free Form Answers
  12. Analysis
  13. $\textsc{thermometer}$ transfers across model scales and benchmarks
  14. $\textsc{thermometer}$ is effective in small labeled data regimes
  15. $\textsc{thermometer}$'s performance improves with number of training tasks
  16. ...and 24 more sections

Key Result

Lemma 4.1

Let $\mathcal{X}$ be the support set of the distribution of ${\bm{x}}$, and assume we have $N_\ast$ i.i.d. samples $\{{\bm{x}}_n\}_{n=1}^{N_\ast}$ from $\mathcal{P}_\ast$. Assume that for fixed parameters $\bm\theta$, $\sup_{{\bm{x}} \in \mathcal{X}} \psi_{\bm\theta}(\phi({\bm{x}})) \leq C_{\bm\thet

Figures (16)

  • Figure 1: Calibration performance against inference runtime. Different methods for calibrating LLaMA-2-Chat 7B compared on the Professional Law task of MMLU on a A100, 40 GB GPU. The task contains $1533$ questions. Our method, $\textsc{thermometer}$ is significantly faster than methods that require multiple forward passes wei-zou-2019-edaxiong2023canjiang2023calibrating at inference time and achieves lower calibration error compared to methods with comparable runtime tian2023just. Vanilla refers to the no-calibration baseline, and TS-CV is a temperature scaling variant (\ref{['sec:exp']}). Similar trends hold for other benchmarks.
  • Figure 2: Evidence accumulation. We illustrate evidence accumulation for the Professional Law dataset from MMLU. It contains $N_k = 1533$ instances. The left panel, shows twenty of the possible $1533$ Gaussians, ${\mathcal{N}}(\tau_k \mid \psi_{\bm\theta}(\phi({\bm{x}}_n^k)), \epsilon = 0.01)$. The right panel, plots the Gaussian proportional to $\prod_{n=1}^{1533}{\mathcal{N}}(\tau_k \mid \psi_{\bm\theta}(\phi({\bm{x}}_n^k)), \epsilon = 0.01)$. The resulting Gaussian is nearly a point mass at $\frac{1}{1533}\sum_{n=1}^{1533}\psi_{\bm\theta}(\phi({\bm{x}}_n^k))$.
  • Figure 3: LLaMA-2-Chat 7B Scatter Plots: ECE Score of 57 MMLU Datasets. The x-axis and y-axis represent ECE score of uncalibared and calibrated model, respectively. $\textsc{thermometer}$ reduces calibration error on all 57 datasets, often substantially.
  • Figure 4: $\textsc{thermometer}$ against temperature scaling on MMLU. Comparison of $\textsc{thermometer}$ predictions and temperatures obtained by temperature scaling. Each green dot represents one MMLU task. The x-coordinate is the temperature learned via temperature scaling and the y-coordinate is the $\textsc{thermometer}$ predicted temperature. $\textsc{thermometer}$ accurately predicts the temperature for unseen task.
  • Figure 5: $\textsc{thermometer}$ transfers across different model scales of LLaMA-2-Chat. We use LLaMA-2-Chat 7B $\textsc{thermometer}$ predicted temperatures for calibrating LLaMA-2-Chat 7B (bold axes), LLaMA-2-Chat 13B and LLaMA-2-Chat 70B. In these plots, each dot represents a MMLU task. The x-coordinate is the ECE achieved by the uncalibrated model, the y-coordinate is the ECE achieved after calibrating the model with $\textsc{thermometer}$. We find that $\textsc{thermometer}$ predicted temperatures from the smaller models also improve calibration of larger models (shown in non-bold axes).
  • ...and 11 more figures

Theorems & Definitions (1)

  • Lemma 4.1