Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Graph Signal Processing Framework for Hallucination Detection in Large Language Models

Valentin Noël

TL;DR

This work introduces a graph signal processing framework to study hallucinations in large language models by modeling transformer layers as attention induced graphs with token embeddings as signals on those graphs. It defines a suite of spectral diagnostics (including layer energy, spectral entropy and high frequency energy ratios) and provides theoretical guarantees linking spectral structure to stability. Across architectures, factual reasoning exhibits universal spectral convergence while hallucinations show distinct fingerprints, enabling principled detection of logical, semantic, and substitution errors. A simple spectral feature based detector achieves 88.75% accuracy, outperforming perplexity-based baselines, demonstrating practical utility for monitoring and debugging LLM outputs. Overall, spectral geometry offers a principled interpretive lens and a diagnostic tool for hallucination detection in large language models.

Abstract

Large language models achieve impressive results but distinguishing factual reasoning from hallucinations remains challenging. We propose a spectral analysis framework that models transformer layers as dynamic graphs induced by attention, with token embeddings as signals on these graphs. Through graph signal processing, we define diagnostics including Dirichlet energy, spectral entropy, and high-frequency energy ratios, with theoretical connections to computational stability. Experiments across GPT architectures suggest universal spectral patterns: factual statements exhibit consistent "energy mountain" behavior with low-frequency convergence, while different hallucination types show distinct signatures. Logical contradictions destabilize spectra with large effect sizes ($g>1.0$), semantic errors remain stable but show connectivity drift, and substitution hallucinations display intermediate perturbations. A simple detector using spectral signatures achieves 88.75% accuracy versus 75% for perplexity-based baselines, demonstrating practical utility. These findings indicate that spectral geometry may capture reasoning patterns and error behaviors, potentially offering a framework for hallucination detection in large language models.

A Graph Signal Processing Framework for Hallucination Detection in Large Language Models

TL;DR

This work introduces a graph signal processing framework to study hallucinations in large language models by modeling transformer layers as attention induced graphs with token embeddings as signals on those graphs. It defines a suite of spectral diagnostics (including layer energy, spectral entropy and high frequency energy ratios) and provides theoretical guarantees linking spectral structure to stability. Across architectures, factual reasoning exhibits universal spectral convergence while hallucinations show distinct fingerprints, enabling principled detection of logical, semantic, and substitution errors. A simple spectral feature based detector achieves 88.75% accuracy, outperforming perplexity-based baselines, demonstrating practical utility for monitoring and debugging LLM outputs. Overall, spectral geometry offers a principled interpretive lens and a diagnostic tool for hallucination detection in large language models.

Abstract

Large language models achieve impressive results but distinguishing factual reasoning from hallucinations remains challenging. We propose a spectral analysis framework that models transformer layers as dynamic graphs induced by attention, with token embeddings as signals on these graphs. Through graph signal processing, we define diagnostics including Dirichlet energy, spectral entropy, and high-frequency energy ratios, with theoretical connections to computational stability. Experiments across GPT architectures suggest universal spectral patterns: factual statements exhibit consistent "energy mountain" behavior with low-frequency convergence, while different hallucination types show distinct signatures. Logical contradictions destabilize spectra with large effect sizes (), semantic errors remain stable but show connectivity drift, and substitution hallucinations display intermediate perturbations. A simple detector using spectral signatures achieves 88.75% accuracy versus 75% for perplexity-based baselines, demonstrating practical utility. These findings indicate that spectral geometry may capture reasoning patterns and error behaviors, potentially offering a framework for hallucination detection in large language models.
Paper Structure (21 sections, 4 theorems, 8 equations, 7 figures, 8 tables)

This paper contains 21 sections, 4 theorems, 8 equations, 7 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Dynamic Attention Graph Model
  3. Graph-signal preliminaries.
  4. Graph-Spectral Diagnostics for LLMs
  5. Theoretical Guarantees
  6. Experimental Results
  7. Cross-Architecture Universality
  8. Spectral Evolution under Factual Reasoning
  9. Hallucination Trajectories
  10. Logical hallucinations.
  11. Semantic hallucinations.
  12. Substitution hallucinations.
  13. Baseline variance contextualization.
  14. Connectivity Drift as Semantic Marker
  15. Baseline Consistency and Statistical Validation
  16. ...and 6 more sections

Key Result

Proposition 1

For any layer $\ell$, In particular, $\mathcal{E}^{(\ell)}=0$ if and only if each column of $X^{(\ell)}$ is constant on the connected component.

Figures (7)

  • Figure 1: Cross-architecture factual baselines. Thin curves: three runs. Thick curves: mean per model. Universality is observed in energy mountain, entropy dip, and smoothness plateau.
  • Figure 2: Logical hallucinations. Transparent: three runs. Thick curves: mean per model. Strong variance emerges in entropy and HFER.
  • Figure 3: Semantic hallucinations. Three runs + mean per model. Variance is minimal, showing spectral stability despite incorrect semantics.
  • Figure 4: Substitution hallucinations. Three runs + mean per model.
  • Figure 5: Fiedler values with baseline error bands. Semantic hallucinations show systematic late-layer drift beyond baseline variability.
  • ...and 2 more figures

Theorems & Definitions (4)

  • Proposition 1: Energy as edge-wise variation
  • Theorem 1: Spectral Poincaré control
  • Proposition 2: High-frequency dominance and local discrepancy
  • Theorem 2: Lipschitz readout under spectral control