Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM Performance Predictors: Learning When to Escalate in Hybrid Human-AI Moderation Systems

Or Bachar, Or Levi, Sardhendu Mishra, Adi Levi, Manpreet Singh Minhas, Justin Miller, Omer Ben-Porat, Eilon Sheetrit, Jonathan Morra

TL;DR

The paper tackles the challenge of when to trust LLM outputs in hybrid human-AI moderation by framing it as a cost-aware decision problem. It introduces LLM Performance Predictors (LPPs), a multifaceted meta-model that fuses gray-box signals (log-probabilities, entropy) with novel uncertainty-attribution indicators and verbalized confidence to predict LLM errors, and then uses a Ridge regression-based controller to decide whether to trust or escalate. Across open and closed LLMs and multimodal multilingual moderation tasks, the LPP framework yields higher predictive accuracy and meaningful cost savings over traditional uncertainty metrics, while enhancing explainability through attribution signals. The work demonstrates practical, scalable uncertainty-aware routing for moderation workflows and provides reproducible code to enable adoption and further research, with implications for broader high-stakes AI-human collaboration domains.

Abstract

As LLMs are increasingly integrated into human-in-the-loop content moderation systems, a central challenge is deciding when their outputs can be trusted versus when escalation for human review is preferable. We propose a novel framework for supervised LLM uncertainty quantification, learning a dedicated meta-model based on LLM Performance Predictors (LPPs) derived from LLM outputs: log-probabilities, entropy, and novel uncertainty attribution indicators. We demonstrate that our method enables cost-aware selective classification in real-world human-AI workflows: escalating high-risk cases while automating the rest. Experiments across state-of-the-art LLMs, including both off-the-shelf (Gemini, GPT) and open-source (Llama, Qwen), on multimodal and multilingual moderation tasks, show significant improvements over existing uncertainty estimators in accuracy-cost trade-offs. Beyond uncertainty estimation, the LPPs enhance explainability by providing new insights into failure conditions (e.g., ambiguous content vs. under-specified policy). This work establishes a principled framework for uncertainty-aware, scalable, and responsible human-AI moderation workflows.

LLM Performance Predictors: Learning When to Escalate in Hybrid Human-AI Moderation Systems

TL;DR

The paper tackles the challenge of when to trust LLM outputs in hybrid human-AI moderation by framing it as a cost-aware decision problem. It introduces LLM Performance Predictors (LPPs), a multifaceted meta-model that fuses gray-box signals (log-probabilities, entropy) with novel uncertainty-attribution indicators and verbalized confidence to predict LLM errors, and then uses a Ridge regression-based controller to decide whether to trust or escalate. Across open and closed LLMs and multimodal multilingual moderation tasks, the LPP framework yields higher predictive accuracy and meaningful cost savings over traditional uncertainty metrics, while enhancing explainability through attribution signals. The work demonstrates practical, scalable uncertainty-aware routing for moderation workflows and provides reproducible code to enable adoption and further research, with implications for broader high-stakes AI-human collaboration domains.

Abstract

As LLMs are increasingly integrated into human-in-the-loop content moderation systems, a central challenge is deciding when their outputs can be trusted versus when escalation for human review is preferable. We propose a novel framework for supervised LLM uncertainty quantification, learning a dedicated meta-model based on LLM Performance Predictors (LPPs) derived from LLM outputs: log-probabilities, entropy, and novel uncertainty attribution indicators. We demonstrate that our method enables cost-aware selective classification in real-world human-AI workflows: escalating high-risk cases while automating the rest. Experiments across state-of-the-art LLMs, including both off-the-shelf (Gemini, GPT) and open-source (Llama, Qwen), on multimodal and multilingual moderation tasks, show significant improvements over existing uncertainty estimators in accuracy-cost trade-offs. Beyond uncertainty estimation, the LPPs enhance explainability by providing new insights into failure conditions (e.g., ambiguous content vs. under-specified policy). This work establishes a principled framework for uncertainty-aware, scalable, and responsible human-AI moderation workflows.
Paper Structure (24 sections, 2 equations, 4 figures, 9 tables)

This paper contains 24 sections, 2 equations, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. LLM Performance Predictors (LPPs)
  5. Prompting
  6. Cost-Aware Escalation Policy
  7. Meta-Model Training
  8. Experiments
  9. Experimental Setup
  10. Datasets
  11. Baselines
  12. Evaluation Metrics
  13. Results
  14. Main Performance Benchmarks: Predictive Accuracy (RQ1)
  15. Cost-Aware Escalation: Operational Efficiency (RQ2)
  16. ...and 9 more sections

Figures (4)

  • Figure 1: A four-stage framework: (1) Base LLM Inference; (2) Integer-Token Output Schema; (3) LPP Feature Extraction; (4a) Meta-Model Training to estimate $s_\theta(x)$; (4b) Cost-Aware Routing to trust or escalate.
  • Figure 2: Ablation study on the OpenAI Moderation Dataset. Bars show total cost ($) when removing one feature family, illustrating each predictor group’s contribution.
  • Figure 3: Ablation study on the Multimodal Moderation Dataset. Bars show total cost ($) when removing one feature family, illustrating each predictor group’s contribution.
  • Figure 4: LPPs as a cross-domain uncertainty router. The meta-model routes between automated decisions and human review using uncertainty signals, applicable to moderation, fraud detection, compliance, and medical triage.