Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generating Faithful Text From a Knowledge Graph with Noisy Reference Text

Tahsina Hashem, Weiqing Wang, Derry Tanti Wijaya, Mohammed Eunus Ali, Yuan-Fang Li

TL;DR

This paper develops a KG-to-text generation model that can generate faithful natural-language text from a given graph, in the presence of noisy reference text, and empower the decoder to control the level of hallucination in the generated text by employing a controllable text generation technique.

Abstract

Knowledge Graph (KG)-to-Text generation aims at generating fluent natural-language text that accurately represents the information of a given knowledge graph. While significant progress has been made in this task by exploiting the power of pre-trained language models (PLMs) with appropriate graph structure-aware modules, existing models still fall short of generating faithful text, especially when the ground-truth natural-language text contains additional information that is not present in the graph. In this paper, we develop a KG-to-text generation model that can generate faithful natural-language text from a given graph, in the presence of noisy reference text. Our framework incorporates two core ideas: Firstly, we utilize contrastive learning to enhance the model's ability to differentiate between faithful and hallucinated information in the text, thereby encouraging the decoder to generate text that aligns with the input graph. Secondly, we empower the decoder to control the level of hallucination in the generated text by employing a controllable text generation technique. We evaluate our model's performance through the standard quantitative metrics as well as a ChatGPT-based quantitative and qualitative analysis. Our evaluation demonstrates the superior performance of our model over state-of-the-art KG-to-text models on faithfulness.

Generating Faithful Text From a Knowledge Graph with Noisy Reference Text

TL;DR

This paper develops a KG-to-text generation model that can generate faithful natural-language text from a given graph, in the presence of noisy reference text, and empower the decoder to control the level of hallucination in the generated text by employing a controllable text generation technique.

Abstract

Knowledge Graph (KG)-to-Text generation aims at generating fluent natural-language text that accurately represents the information of a given knowledge graph. While significant progress has been made in this task by exploiting the power of pre-trained language models (PLMs) with appropriate graph structure-aware modules, existing models still fall short of generating faithful text, especially when the ground-truth natural-language text contains additional information that is not present in the graph. In this paper, we develop a KG-to-text generation model that can generate faithful natural-language text from a given graph, in the presence of noisy reference text. Our framework incorporates two core ideas: Firstly, we utilize contrastive learning to enhance the model's ability to differentiate between faithful and hallucinated information in the text, thereby encouraging the decoder to generate text that aligns with the input graph. Secondly, we empower the decoder to control the level of hallucination in the generated text by employing a controllable text generation technique. We evaluate our model's performance through the standard quantitative metrics as well as a ChatGPT-based quantitative and qualitative analysis. Our evaluation demonstrates the superior performance of our model over state-of-the-art KG-to-text models on faithfulness.
Paper Structure (27 sections, 5 equations, 10 figures, 6 tables)

This paper contains 27 sections, 5 equations, 10 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Knowledge Graph-to-Text Generation
  4. Controlling Hallucinations in Text Generation
  5. Evaluation using ChatGPT
  6. Proposed Model
  7. Problem Formulation
  8. Our Framework
  9. Minimizing Hallucinations with Contrastive Learning
  10. Controlling Hallucinations with Control Feature Token
  11. Experiments
  12. Dataset
  13. House.
  14. GenWiki.
  15. Baseline Models
  16. ...and 12 more sections

Figures (10)

  • Figure 1: A sample knowledge graph for the House dataset with its ground-truth text. The red colored text in the ground-truth text represents extrinsic hallucination information.
  • Figure 2: The overall framework of our KG-to-text model.
  • Figure 3: Controllable Text generation with Control Feature Token
  • Figure 4: ChatGPT-based evaluation on $50$ samples from the House test set.
  • Figure 5: Prompt templates for enumerating facts using ChatGPT. Template-1 (left) is to enumerate facts in the input (linearized graph). Template-2 (middle) is to enumerate common facts between the input (linearized graph) and the output (generated text). Template-3 (right) is to enumerate hallucinated facts in the output (generated text).
  • ...and 5 more figures