Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Models Know When They Do Not Know: Calibration, Cascading, and Cleaning

Chenjie Hao, Weyl Lu, Yuko Ishiwaka, Zengyi Li, Weier Wan, Yubei Chen

TL;DR

The paper tackles the problem of models recognizing when they do not know by proposing a unified calibration framework that turns internal signals into calibrated confidence scores $c_M(X,Y)$. It introduces temperature scaling and Platt scaling as training-free calibration tools and shows how calibrated confidence enables two practical apps: confidence-based cascading (including small–large and large–large cascades) to improve efficiency without sacrificing accuracy, and a mixture-of-experts data cleaning approach that uses ensemble agreement to identify mislabeled data in ImageNet-1K and MMLU. The key findings show that calibrated confidence is reliable across validation and test sets, correlates with accuracy, and that cascades and cleaning outperform baselines and scale to OOD conditions. Together, these results provide a practical, training-free pathway to more efficient, reliable, and trustworthy AI systems across both vision and language domains.

Abstract

When a model knows when it does not know, many possibilities emerge. The first question is how to enable a model to recognize that it does not know. A promising approach is to use confidence, computed from the model's internal signals, to reflect its ignorance. Prior work in specific domains has shown that calibration can provide reliable confidence estimates. In this work, we propose a simple, effective, and universal training-free method that applies to both vision and language models, performing model calibration, cascading, and data cleaning to better exploit a model's ability to recognize when it does not know. We first highlight two key empirical observations: higher confidence corresponds to higher accuracy within a single model, and models calibrated on the validation set remain calibrated on a held-out test set. These findings empirically establish the reliability and comparability of calibrated confidence. Building on this, we introduce two applications: (1) model cascading with calibrated advantage routing and (2) data cleaning based on model ensemble. Using the routing signal derived from the comparability of calibrated confidences, we cascade large and small models to improve efficiency with almost no compromise in accuracy, and we further cascade two models of comparable scale to achieve performance beyond either model alone. Leveraging multiple experts and their calibrated confidences, we design a simple yet effective data-cleaning method that balances precision and detection rate to identify mislabeled samples in ImageNet and Massive Multitask Language Understanding (MMLU) datasets. Our results demonstrate that enabling models to recognize when they do not know is a practical step toward more efficient, reliable, and trustworthy AI.

When Models Know When They Do Not Know: Calibration, Cascading, and Cleaning

TL;DR

The paper tackles the problem of models recognizing when they do not know by proposing a unified calibration framework that turns internal signals into calibrated confidence scores . It introduces temperature scaling and Platt scaling as training-free calibration tools and shows how calibrated confidence enables two practical apps: confidence-based cascading (including small–large and large–large cascades) to improve efficiency without sacrificing accuracy, and a mixture-of-experts data cleaning approach that uses ensemble agreement to identify mislabeled data in ImageNet-1K and MMLU. The key findings show that calibrated confidence is reliable across validation and test sets, correlates with accuracy, and that cascades and cleaning outperform baselines and scale to OOD conditions. Together, these results provide a practical, training-free pathway to more efficient, reliable, and trustworthy AI systems across both vision and language domains.

Abstract

When a model knows when it does not know, many possibilities emerge. The first question is how to enable a model to recognize that it does not know. A promising approach is to use confidence, computed from the model's internal signals, to reflect its ignorance. Prior work in specific domains has shown that calibration can provide reliable confidence estimates. In this work, we propose a simple, effective, and universal training-free method that applies to both vision and language models, performing model calibration, cascading, and data cleaning to better exploit a model's ability to recognize when it does not know. We first highlight two key empirical observations: higher confidence corresponds to higher accuracy within a single model, and models calibrated on the validation set remain calibrated on a held-out test set. These findings empirically establish the reliability and comparability of calibrated confidence. Building on this, we introduce two applications: (1) model cascading with calibrated advantage routing and (2) data cleaning based on model ensemble. Using the routing signal derived from the comparability of calibrated confidences, we cascade large and small models to improve efficiency with almost no compromise in accuracy, and we further cascade two models of comparable scale to achieve performance beyond either model alone. Leveraging multiple experts and their calibrated confidences, we design a simple yet effective data-cleaning method that balances precision and detection rate to identify mislabeled samples in ImageNet and Massive Multitask Language Understanding (MMLU) datasets. Our results demonstrate that enabling models to recognize when they do not know is a practical step toward more efficient, reliable, and trustworthy AI.
Paper Structure (32 sections, 10 equations, 8 figures, 7 tables, 2 algorithms)

This paper contains 32 sections, 10 equations, 8 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Method
  3. On calibration of vision model and language model
  4. Formulation.
  5. Temperature Scaling.
  6. Platt Scaling.
  7. Cascading with Calibrated Advantage Routing
  8. Data Cleaning with Model Ensemble
  9. Basic Cleaning.
  10. Cleaning with Calibration.
  11. Confident Learning.
  12. Experiments
  13. calibration can be reliable
  14. Cascade Small and Large Models
  15. Cascading Same-Size Models
  16. ...and 17 more sections

Figures (8)

  • Figure 1: ImageNet-1K (vision domain)
  • Figure 2: MMLU (language domain)
  • Figure 4: Cascade Results on ImageNet-1K. Each panel shows cascade accuracy vs. the routing ratio $p$. Panel (a) uses a small–large model pair, while panels (b) and (c) combine models of the same size, with panel (b) using a medium–medium pair and panel (c) using a large–large pair.
  • Figure 5: Cascade Results on Language Models. Panels (a)–(e) show small–large model cascades on five datasets. Panel (f) shows a large–large cascade, where $p$ denotes the fraction of samples routed to the first large model.
  • Figure 6: OOD evaluation on ImageNet-C hendrycks2019benchmarking with all corruption types at severity level 5. Results for severity levels 1 and 3 are provided in the appendix.
  • ...and 3 more figures