Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Chance-Constrained Inference for Hallucination Risk Control in Large Language Models

Sreenivasan Mohandas

TL;DR

Chance-constrained inference (CCI) reframes hallucination risk in large language models as a deployment-time probabilistic constraint, bounding the conditional violation rate among accepted outputs under stochastic decoding. It introduces a sequential, anytime-valid feasibility-certification procedure that adapts the number of samples and requires no retraining, calibration, or external retraining of the model. Empirical results on natural-language question answering tasks show reliable risk control, early detection of intrinsically infeasible inputs, and safe composition under repeated use, outperforming confidence-based selective baselines which fail to provide guarantees. The framework supports severity-weighted and hierarchical constraints and delivers output-level guarantees through abstention, offering practical reliability for real-world, stochastic generation settings.

Abstract

Large language models generate outputs stochastically and may produce fluent but invalid responses, including factual hallucinations. Existing mitigation strategies reduce average error rates but do not provide explicit control over the \emph{frequency} of such failures under repeated use. We formulate inference as a deployment-time risk control problem and introduce \emph{chance-constrained inference}, which directly bounds the probability of hallucinations among accepted generations. Hallucinations are modeled as stochastic constraint violations, and we show that confidence-based selective prediction does not, in general, imply probabilistic risk guarantees. To enforce chance constraints efficiently, we propose a sequential, anytime-valid inference procedure that adaptively certifies feasibility or infeasibility using finite samples, avoiding conservative fixed-sample bounds. Experiments on questions inspired by NaturalQuestions and controlled multi-hop question answering demonstrate reliable risk control, early detection of intrinsically infeasible inputs, and safe composition under repeated use, while confidence-based baselines fail to provide consistent guarantees.

Chance-Constrained Inference for Hallucination Risk Control in Large Language Models

TL;DR

Chance-constrained inference (CCI) reframes hallucination risk in large language models as a deployment-time probabilistic constraint, bounding the conditional violation rate among accepted outputs under stochastic decoding. It introduces a sequential, anytime-valid feasibility-certification procedure that adapts the number of samples and requires no retraining, calibration, or external retraining of the model. Empirical results on natural-language question answering tasks show reliable risk control, early detection of intrinsically infeasible inputs, and safe composition under repeated use, outperforming confidence-based selective baselines which fail to provide guarantees. The framework supports severity-weighted and hierarchical constraints and delivers output-level guarantees through abstention, offering practical reliability for real-world, stochastic generation settings.

Abstract

Large language models generate outputs stochastically and may produce fluent but invalid responses, including factual hallucinations. Existing mitigation strategies reduce average error rates but do not provide explicit control over the \emph{frequency} of such failures under repeated use. We formulate inference as a deployment-time risk control problem and introduce \emph{chance-constrained inference}, which directly bounds the probability of hallucinations among accepted generations. Hallucinations are modeled as stochastic constraint violations, and we show that confidence-based selective prediction does not, in general, imply probabilistic risk guarantees. To enforce chance constraints efficiently, we propose a sequential, anytime-valid inference procedure that adaptively certifies feasibility or infeasibility using finite samples, avoiding conservative fixed-sample bounds. Experiments on questions inspired by NaturalQuestions and controlled multi-hop question answering demonstrate reliable risk control, early detection of intrinsically infeasible inputs, and safe composition under repeated use, while confidence-based baselines fail to provide consistent guarantees.
Paper Structure (36 sections, 3 theorems, 42 equations, 2 tables, 1 algorithm)

This paper contains 36 sections, 3 theorems, 42 equations, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Notation and Problem Setup
  4. Constraint Modeling and Hallucination
  5. Chance-Constrained Inference
  6. Conditional Risk and Acceptance.
  7. Inference as Feasibility Certification.
  8. Operational Acceptance Policy.
  9. Feasibility Gap.
  10. Limits of Confidence-Based Selective Prediction.
  11. Conceptual Contribution.
  12. Utility--Risk Tradeoffs (Discussion)
  13. Finite-Sample Risk Estimation and Guarantees
  14. Independence assumption.
  15. Severity-Weighted and Hierarchical Constraints
  16. ...and 21 more sections

Key Result

Theorem 1

Fix an input $x$, risk budget $\epsilon(x)$, and confidence level $\delta$. Let Algorithm alg:cci terminate at a (random) stopping time $\tau$. With probability at least $1-\delta$: These guarantees hold under arbitrary data-dependent stopping.

Theorems & Definitions (5)

  • Theorem 1: Anytime-Valid Feasibility Certification
  • Theorem 2: Time-Uniform Feasibility Certification
  • proof : Proof sketch
  • Lemma 1: Conditional Risk Control
  • proof