Detecting Hallucinations in Retrieval-Augmented Generation via Semantic-level Internal Reasoning Graph

TL;DR

Faithfulness hallucinations in retrieval-augmented generation remain a challenge despite factual grounding. The paper introduces SIRG, a semantic-level internal reasoning graph built by extending token-level Layer-wise Relevance Propagation to semantic fragments of context and response, and trains a lightweight AlignScore RoBERTa-based discriminator to detect hallucinations at the fragment level. It constructs a semantic attribution matrix $W$ using Top-k or Adaptive edge selection and uses a threshold $\\alpha$ to determine overall hallucination, achieving state-of-the-art performance on RAGTruth and Dolly-15k while using a lightweight classifier. This approach improves interpretability by tying decisions to internal reasoning fragments and reduces computational costs compared to multi-round LLM-based verifications, with potential for adaptable security-sensitive deployments. $

Abstract

The Retrieval-augmented generation (RAG) system based on Large language model (LLM) has made significant progress. It can effectively reduce factuality hallucinations, but faithfulness hallucinations still exist. Previous methods for detecting faithfulness hallucinations either neglect to capture the models' internal reasoning processes or handle those features coarsely, making it difficult for discriminators to learn. This paper proposes a semantic-level internal reasoning graph-based method for detecting faithfulness hallucination. Specifically, we first extend the layer-wise relevance propagation algorithm from the token level to the semantic level, constructing an internal reasoning graph based on attribution vectors. This provides a more faithful semantic-level representation of dependency. Furthermore, we design a general framework based on a small pre-trained language model to utilize the dependencies in LLM's reasoning for training and hallucination detection, which can dynamically adjust the pass rate of correct samples through a threshold. Experimental results demonstrate that our method achieves better overall performance compared to state-of-the-art baselines on RAGTruth and Dolly-15k.

Figures (9)

• Figure 1: Example of a semantic-level internal reasoning graph. Yellow nodes represent contextual semantic fragments, blue nodes represent semantic fragments of the model's response, and the weight on the edge indicates the contribution degree of the source semantic fragment to the target (with an upper boundary of 1). The dashed box indicates a hallucinated semantic fragment.
• Figure 2: The framework of our method. LRP is first applied to derive a relevance distribution $R_i$ based on the parameters of the LLM. Based on $R_i$ of entities and semantic fragments of input and output content, a semantic-level internal reasoning graph of LLMs is constructed. Subsequently, a PLM is used to determine whether each fragment exhibits semantic conflicts or omissions. Finally, the degree of hallucination occurring in the reasoning graph units is used to determine whether the model's overall response exhibits hallucinations.
• Figure 3: Perturbation tests on RAGTruth with Llama-7B. $(y_0-y_p)^2$ indicates the change of final embeddings before and after perturbation, while $logit s_k$ represents the average probability of the target semantic fragment. The dashed line shows the curve state after random addition or pruning. Perturbation tests are conducted on 100 samples and mean of the above indicators are took.
• Figure 4: Overall precision, recall, and F1 score of Llama-13B on RAGTruth are evaluated by setting different $\alpha$ values.
• Figure 5: The attribution of LLM responses is obtained by processing attribution features at a coarse granularity. The left figure shows the maximum value of the attribution vector for all tokens in the response, while the right figure shows the average value. The deeper the red color, the greater the token's contribution to the target response.