The Effects of Hallucinations in Synthetic Training Data for Relation Extraction

TL;DR

The study addresses how hallucinations in synthetic training data from Generative Data Augmentation degrade relation extraction, especially at the document level where recall can drop by up to ~39%. It differentiates between relevant and irrelevant hallucinations, showing that relevant ones drive most performance loss, while irrelevant ones have limited impact. The authors develop two detection approaches—NER-based and textual entailment—that achieve high F1 scores (83.8% and 92.2%, respectively) to identify and mitigate hallucinations, aiding data cleaning and prevalence estimation. The findings underscore the need to filter hallucinations in GDA pipelines to preserve RE quality and propose practical strategies for data curation and evaluation in RE tasks.

Abstract

Relation extraction is crucial for constructing knowledge graphs, with large high-quality datasets serving as the foundation for training, fine-tuning, and evaluating models. Generative data augmentation (GDA) is a common approach to expand such datasets. However, this approach often introduces hallucinations, such as spurious facts, whose impact on relation extraction remains underexplored. In this paper, we examine the effects of hallucinations on the performance of relation extraction on the document and sentence levels. Our empirical study reveals that hallucinations considerably compromise the ability of models to extract relations from text, with recall reductions between 19.1% and 39.2%. We identify that relevant hallucinations impair the model's performance, while irrelevant hallucinations have a minimal impact. Additionally, we develop methods for the detection of hallucinations to improve data quality and model performance. Our approaches successfully classify texts as either 'hallucinated' or 'clean,' achieving high F1-scores of 83.8% and 92.2%. These methods not only assist in removing hallucinations but also help in estimating their prevalence within datasets, which is crucial for selecting high-quality data. Overall, our work confirms the profound impact of relevant hallucinations on the effectiveness of relation extraction models.

Figures (5)

• Figure 1: Text generated by Generative Data Augmentation (GDA) with and without hallucinations.
• Figure 2: Examples of hallucinations, showing the distinction between relevant hallucinations (e.g., relation type 'birthDate') and irrelevant hallucinations (e.g., relation type 'occupation').
• Figure 3: Illustration of the approach used for testing the influence of hallucinations at document-level with the datasets A and B and the language model approach DREEAM.
• Figure 4: Illustration of how LLAMA2-13b-chat may introduce hallucinations into a text.
• Figure 5: Precision, recall, and F1-score of the NER metric at varying acceptance thresholds. The highest F1-score is achieved at a threshold of 0.55.