Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CORVUS: Red-Teaming Hallucination Detectors via Internal Signal Camouflage in Large Language Models

Nay Myat Min, Long H. Pham, Hongyu Zhang, Jun Sun

TL;DR

Single-pass hallucination detectors rely on internal telemetry, but this work demonstrates a model-side vulnerability: a lightweight adapter-based red team (CORVUS) can camouflage telemetry signals (TE, HV, AD) to erode detector effectiveness while preserving surface answers. CORVUS uses a teacher-forced replay with an adversarial embedding FGSM step to shift telemetry, training only LoRA adapters to degrade detector separability without dulling generation. The method transfers from 1k Alpaca instructions to FAVA-Annotation across four open-weight LLMs, consistently degrading both training-free detectors (e.g., LLM-Check) and probes (SEP, ICR-probe). The results underscore that relying solely on internal telemetry is insufficient for robust auditing and motivate integrating external grounding or cross-model evidence for safer, trustworthy AI systems.

Abstract

Single-pass hallucination detectors rely on internal telemetry (e.g., uncertainty, hidden-state geometry, and attention) of large language models, implicitly assuming hallucinations leave separable traces in these signals. We study a white-box, model-side adversary that fine-tunes lightweight LoRA adapters on the model while keeping the detector fixed, and introduce CORVUS, an efficient red-teaming procedure that learns to camouflage detector-visible telemetry under teacher forcing, including an embedding-space FGSM attention stress test. Trained on 1,000 out-of-distribution Alpaca instructions (<0.5% trainable parameters), CORVUS transfers to FAVA-Annotation across Llama-2, Vicuna, Llama-3, and Qwen2.5, and degrades both training-free detectors (e.g., LLM-Check) and probe-based detectors (e.g., SEP, ICR-probe), motivating adversary-aware auditing that incorporates external grounding or cross-model evidence.

CORVUS: Red-Teaming Hallucination Detectors via Internal Signal Camouflage in Large Language Models

TL;DR

Single-pass hallucination detectors rely on internal telemetry, but this work demonstrates a model-side vulnerability: a lightweight adapter-based red team (CORVUS) can camouflage telemetry signals (TE, HV, AD) to erode detector effectiveness while preserving surface answers. CORVUS uses a teacher-forced replay with an adversarial embedding FGSM step to shift telemetry, training only LoRA adapters to degrade detector separability without dulling generation. The method transfers from 1k Alpaca instructions to FAVA-Annotation across four open-weight LLMs, consistently degrading both training-free detectors (e.g., LLM-Check) and probes (SEP, ICR-probe). The results underscore that relying solely on internal telemetry is insufficient for robust auditing and motivate integrating external grounding or cross-model evidence for safer, trustworthy AI systems.

Abstract

Single-pass hallucination detectors rely on internal telemetry (e.g., uncertainty, hidden-state geometry, and attention) of large language models, implicitly assuming hallucinations leave separable traces in these signals. We study a white-box, model-side adversary that fine-tunes lightweight LoRA adapters on the model while keeping the detector fixed, and introduce CORVUS, an efficient red-teaming procedure that learns to camouflage detector-visible telemetry under teacher forcing, including an embedding-space FGSM attention stress test. Trained on 1,000 out-of-distribution Alpaca instructions (<0.5% trainable parameters), CORVUS transfers to FAVA-Annotation across Llama-2, Vicuna, Llama-3, and Qwen2.5, and degrades both training-free detectors (e.g., LLM-Check) and probe-based detectors (e.g., SEP, ICR-probe), motivating adversary-aware auditing that incorporates external grounding or cross-model evidence.
Paper Structure (53 sections, 9 equations, 1 figure, 5 tables, 1 algorithm)

This paper contains 53 sections, 9 equations, 1 figure, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background: Hallucination Detection
  3. Training-free defenses.
  4. Training-based defenses.
  5. Retrieval-augmented defenses.
  6. The Proposed CORVUS Method
  7. Method overview
  8. Threat model
  9. Terminology
  10. Telemetry over internal signals
  11. Token entropy (TE).
  12. Hidden log-volume (HV).
  13. Attention diagonality (AD).
  14. Detector assumptions and signals
  15. Observation.
  16. ...and 38 more sections

Figures (1)

  • Figure 1: CORVUS method.(1) A prompt and fixed answer are processed under teacher forcing with trainable LoRA adapters. (2) A clean pass yields logits, hidden states, and attention over the answer window. (3) Telemetry features are extracted: TE, HV, and AD. (4) An AD-targeted single-step FGSM embedding perturbation induces a perturbed pass to obtain $AD_{\mathrm{adv}}$; a combined loss updates only LoRA to reshape detector-relevant internal signals.