Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Entropy Sentinel: Continuous LLM Accuracy Monitoring from Decoding Entropy Traces in STEM

Pedro Memoli Buffa, Luciano Del Corro

TL;DR

This work investigates continuous monitoring of LLM accuracy under domain shift by leveraging inference-time entropy traces derived from final-layer top-$k$ log-probabilities. By compressing each generation's entropy into an 11-dimensional feature vector and training a lightweight probabilistic predictor, the method yields instance-level correctness probabilities that are averaged to produce domain-level accuracy estimates. Across ten STEM benchmarks and nine LLMs, entropy-profile signals often track held-out accuracy and preserve domain rankings, with reliability strongly influenced by the diversity of supervision used during training; mixing easy and hard tasks generally yields the best generalization. The approach offers a scalable, deployment-friendly primitive for monitoring and data acquisition prioritization, compatible with both open and closed models, though practitioners should validate calibration on their target model due to model-dependence and potential calibration gaps.

Abstract

Deploying LLMs raises two coupled challenges: (1) monitoring - estimating where a model underperforms as traffic and domains drift - and (2) improvement - prioritizing data acquisition to close the largest performance gaps. We test whether an inference-time signal can estimate slice-level accuracy under domain shift. For each response, we compute an output-entropy profile from final-layer next-token probabilities (from top-k logprobs) and summarize it with eleven statistics. A lightweight classifier predicts instance correctness, and averaging predicted probabilities yields a domain-level accuracy estimate. We evaluate on ten STEM reasoning benchmarks with exhaustive train/test compositions (k in {1,2,3,4}; all "10 choose k" combinations), across nine LLMs from six families (3B-20B). Estimates often track held-out benchmark accuracy, and several models show near-monotonic ordering of domains. Output-entropy profiles are thus an accessible signal for scalable monitoring and for targeting data acquisition.

Entropy Sentinel: Continuous LLM Accuracy Monitoring from Decoding Entropy Traces in STEM

TL;DR

This work investigates continuous monitoring of LLM accuracy under domain shift by leveraging inference-time entropy traces derived from final-layer top- log-probabilities. By compressing each generation's entropy into an 11-dimensional feature vector and training a lightweight probabilistic predictor, the method yields instance-level correctness probabilities that are averaged to produce domain-level accuracy estimates. Across ten STEM benchmarks and nine LLMs, entropy-profile signals often track held-out accuracy and preserve domain rankings, with reliability strongly influenced by the diversity of supervision used during training; mixing easy and hard tasks generally yields the best generalization. The approach offers a scalable, deployment-friendly primitive for monitoring and data acquisition prioritization, compatible with both open and closed models, though practitioners should validate calibration on their target model due to model-dependence and potential calibration gaps.

Abstract

Deploying LLMs raises two coupled challenges: (1) monitoring - estimating where a model underperforms as traffic and domains drift - and (2) improvement - prioritizing data acquisition to close the largest performance gaps. We test whether an inference-time signal can estimate slice-level accuracy under domain shift. For each response, we compute an output-entropy profile from final-layer next-token probabilities (from top-k logprobs) and summarize it with eleven statistics. A lightweight classifier predicts instance correctness, and averaging predicted probabilities yields a domain-level accuracy estimate. We evaluate on ten STEM reasoning benchmarks with exhaustive train/test compositions (k in {1,2,3,4}; all "10 choose k" combinations), across nine LLMs from six families (3B-20B). Estimates often track held-out benchmark accuracy, and several models show near-monotonic ordering of domains. Output-entropy profiles are thus an accessible signal for scalable monitoring and for targeting data acquisition.
Paper Structure (35 sections, 3 equations, 5 figures, 10 tables)

This paper contains 35 sections, 3 equations, 5 figures, 10 tables.

Table of Contents

  1. Introduction
  2. From Signatures to Accuracy Estimates
  3. Entropy Profile Signals
  4. Accuracy Estimation via Entropy Profile Features
  5. Experimental Setup
  6. Results
  7. RQ1: Accuracy estimation under deployment-plausible defaults.
  8. RQ2: Baseline UQ metrics for performance monitoring.
  9. RQ3: Training composition dominates (and improves with $k$).
  10. RQ4: Estimator choices matter, but less than data composition.
  11. Practical takeaways.
  12. Related Work
  13. Conclusion
  14. Limitations
  15. Baseline White Box UQ Metrics
  16. ...and 20 more sections

Figures (5)

  • Figure 1: Entropy-based accuracy estimation for PHI-3.5-MINI-3.6B. Trained on two benchmarks (orange), the estimator generalizes to eight unseen STEM benchmarks (blue)
  • Figure 2: Max-entropy density for phi-3.5-mini on MATH (correct vs. incorrect). Incorrect responses shift to higher entropy, indicating greater uncertainty.
  • Figure 3: Accuracy estimations from a random-forest classifier trained exclusively on compact entropy-profile features on GSM and OlympiadBench. Both train benchmarks span the two extremes of difficulty.
  • Figure 4: Training-group difficulty vs. estimation quality for Phi-3.5-Mini: intermediate weighted accuracy (0.4--0.6) yields the lowest AEE; all-easy/all-hard groups perform worse.
  • Figure 5: Relationship between training group difficulty and estimation quality aggregated across all nine LLMs. Training groups with intermediate weighted accuracy (0.4--0.6) achieve optimal performance, while difficulty-homogeneous groups at either extreme degrade generalization. Shaded region indicates IQR.