Temporal Graph Network: Hallucination Detection in Multi-Turn Conversation

TL;DR

The paper introduces a Temporal Graph Network (TGN) for dialogue-level hallucination detection in multi-turn conversations by modeling each turn as a node and connecting turns via temporal and shared-entity edges. Through sentence-transformer node embeddings, graph neural network message passing, and attention pooling, the approach yields a graph-level representation fed to a classifier that distinguishes six hallucination types and provides interpretable attention-based justifications. Compared to a strong GCA baseline, TGN variants, especially those combining temporal and entity edges, achieve improved accuracy while maintaining efficiency with lightweight encoders. The work offers a scalable, explainable framework for detecting inconsistencies across dialogue histories and suggests promising directions for incorporating richer relations and external knowledge for enhanced reliability in real-world systems.

Abstract

Hallucinations can be produced by conversational AI systems, particularly in multi-turn conversations where context changes and contradictions may eventually surface. By representing the entire conversation as a temporal graph, we present a novel graph-based method for detecting dialogue-level hallucinations. Our framework models each dialogue as a node, encoding it using a sentence transformer. We explore two different ways of connectivity: i) shared-entity edges, which connect turns that refer to the same entities; ii) temporal edges, which connect contiguous turns in the conversation. Message-passing is used to update the node embeddings, allowing flow of information between related nodes. The context-aware node embeddings are then combined using attention pooling into a single vector, which is then passed on to a classifier to determine the presence and type of hallucinations. We demonstrate that our method offers slightly improved performance over existing methods. Further, we show the attention mechanism can be used to justify the decision making process. The code and model weights are made available at: https://github.com/sambuaneesh/anlp-project.

Figures (4)

• Figure 1: A multi-turn conversation where context evolves and contradictions emerge over time, illustrating dialogue-level hallucination.
• Figure 2: Dialogue as a temporal graph. Temporal edges connect adjacent turns, while shared-entity edges link utterances referencing the same entities. Message passing and attention pooling enable dialogue-level hallucination detection.
• Figure 3: Overview of the framework. Each dialogue turn is encoded into a sentence embedding and represented as a node in a temporal graph. Temporal edges connect sequential turns, while shared-entity edges link dialogues that mention the same entities. A message-passing GNN propagates contextual information across turns, and attention pooling aggregates the node embeddings into a dialogue-level representation. The final classifier predicts one of six hallucination types.
• Figure 4: Confusion matrix showing the distribution of model predictions across hallucination categories. It highlights different failure modes and the model’s ability to correctly identify non-hallucinatory instances.