Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency

Jiaxin Zhang, Zhuohang Li, Kamalika Das, Bradley A. Malin, Sricharan Kumar

TL;DR

This paper shows that relying solely on self-consistency to detect language model hallucinations is insufficient, revealing both question-level and model-level failure modes. It introduces SAC3, a semantic-aware cross-checking framework that combines semantically equivalent question perturbations with cross-model verifier checks, organized into three stages and a composite scoring system. Across classification and open-domain QA benchmarks, SAC3 significantly outperforms self-consistency baselines, delivering high AUROC and robust performance under varying thresholds and model types. The approach offers a practical, parallelizable method for improving factuality assessments of black-box LMs with potential for broad application in QA and generation tasks.

Abstract

Hallucination detection is a critical step toward understanding the trustworthiness of modern language models (LMs). To achieve this goal, we re-examine existing detection approaches based on the self-consistency of LMs and uncover two types of hallucinations resulting from 1) question-level and 2) model-level, which cannot be effectively identified through self-consistency check alone. Building upon this discovery, we propose a novel sampling-based method, i.e., semantic-aware cross-check consistency (SAC3) that expands on the principle of self-consistency checking. Our SAC3 approach incorporates additional mechanisms to detect both question-level and model-level hallucinations by leveraging advances including semantically equivalent question perturbation and cross-model response consistency checking. Through extensive and systematic empirical analysis, we demonstrate that SAC3 outperforms the state of the art in detecting both non-factual and factual statements across multiple question-answering and open-domain generation benchmarks.

SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency

TL;DR

This paper shows that relying solely on self-consistency to detect language model hallucinations is insufficient, revealing both question-level and model-level failure modes. It introduces SAC3, a semantic-aware cross-checking framework that combines semantically equivalent question perturbations with cross-model verifier checks, organized into three stages and a composite scoring system. Across classification and open-domain QA benchmarks, SAC3 significantly outperforms self-consistency baselines, delivering high AUROC and robust performance under varying thresholds and model types. The approach offers a practical, parallelizable method for improving factuality assessments of black-box LMs with potential for broad application in QA and generation tasks.

Abstract

Hallucination detection is a critical step toward understanding the trustworthiness of modern language models (LMs). To achieve this goal, we re-examine existing detection approaches based on the self-consistency of LMs and uncover two types of hallucinations resulting from 1) question-level and 2) model-level, which cannot be effectively identified through self-consistency check alone. Building upon this discovery, we propose a novel sampling-based method, i.e., semantic-aware cross-check consistency (SAC3) that expands on the principle of self-consistency checking. Our SAC3 approach incorporates additional mechanisms to detect both question-level and model-level hallucinations by leveraging advances including semantically equivalent question perturbation and cross-model response consistency checking. Through extensive and systematic empirical analysis, we demonstrate that SAC3 outperforms the state of the art in detecting both non-factual and factual statements across multiple question-answering and open-domain generation benchmarks.
Paper Structure (42 sections, 8 equations, 6 figures, 7 tables)

This paper contains 42 sections, 8 equations, 6 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Key Observations.
  3. Proposed Approach.
  4. Effectiveness of SAC$^3$.
  5. Related Work
  6. Consistency Evaluation of LMs.
  7. Self-consistency Limitations in Factuality Assessment
  8. SAC$^3$ : Semantic-Aware Cross-check Consistency
  9. Stage 1: Question-level Cross-checking via Semantically Equivalent Perturbations
  10. Stage 2: Model-level Cross-check with Additional Verifier LM
  11. Stage 3: Consistency Score Calculation
  12. Semantic-aware Consistency Check of QA Pairs
  13. Self-checking Consistency (SC$^2$) Score
  14. Question-level Consistency (SAC$^3$-Q) Score
  15. Model-level Consistency (SAC$^3$-M & SAC$^3$-QM) Score
  16. ...and 27 more sections

Figures (6)

  • Figure 1: Key observation: solely checking the self-consistency of LLMs is not sufficient for deciding factuality. Left: generated responses to the same question may be consistent but non-factual. Right: generated responses may be inconsistent with the original answer that is factually correct.
  • Figure 2: Overview of the proposed semantic-aware cross-check consistency (SAC$^3$) method.
  • Figure 3: Impact of threshold on detection accuracy.
  • Figure 4: Histogram of hallucination score.
  • Figure 5: Effect of verifier LM weight on AUROC.
  • ...and 1 more figures