Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graphical Reasoning: LLM-based Semi-Open Relation Extraction

Yicheng Tao, Yiqun Wang, Longju Bai

TL;DR

This work tackles relation extraction with large language models by combining Chain of Thought with In-Context Learning and a novel Graphical Reasoning framework that decomposes RE into entity extraction, paraphrase, and relation validation. It uses 13 in-context CoT examples for GPT-3.5 and a GRE formulation with a probabilistic objective that leverages paraphrased text. The authors create a manually annotated version of CoNLL04 to improve evaluation reliability and release the dataset. Empirical results on ADE, CoNLL04, and NYT show GRE and CoT benefits, with GRE outperforming CoT on several datasets and improved data quality boosting performance.

Abstract

This paper presents a comprehensive exploration of relation extraction utilizing advanced language models, specifically Chain of Thought (CoT) and Graphical Reasoning (GRE) techniques. We demonstrate how leveraging in-context learning with GPT-3.5 can significantly enhance the extraction process, particularly through detailed example-based reasoning. Additionally, we introduce a novel graphical reasoning approach that dissects relation extraction into sequential sub-tasks, improving precision and adaptability in processing complex relational data. Our experiments, conducted on multiple datasets, including manually annotated data, show considerable improvements in performance metrics, underscoring the effectiveness of our methodologies.

Graphical Reasoning: LLM-based Semi-Open Relation Extraction

TL;DR

This work tackles relation extraction with large language models by combining Chain of Thought with In-Context Learning and a novel Graphical Reasoning framework that decomposes RE into entity extraction, paraphrase, and relation validation. It uses 13 in-context CoT examples for GPT-3.5 and a GRE formulation with a probabilistic objective that leverages paraphrased text. The authors create a manually annotated version of CoNLL04 to improve evaluation reliability and release the dataset. Empirical results on ADE, CoNLL04, and NYT show GRE and CoT benefits, with GRE outperforming CoT on several datasets and improved data quality boosting performance.

Abstract

This paper presents a comprehensive exploration of relation extraction utilizing advanced language models, specifically Chain of Thought (CoT) and Graphical Reasoning (GRE) techniques. We demonstrate how leveraging in-context learning with GPT-3.5 can significantly enhance the extraction process, particularly through detailed example-based reasoning. Additionally, we introduce a novel graphical reasoning approach that dissects relation extraction into sequential sub-tasks, improving precision and adaptability in processing complex relational data. Our experiments, conducted on multiple datasets, including manually annotated data, show considerable improvements in performance metrics, underscoring the effectiveness of our methodologies.
Paper Structure (24 sections, 2 equations, 1 figure, 1 table)

This paper contains 24 sections, 2 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Relation Extraction with LLMs
  4. Prompt Engineering for LLMs
  5. Method
  6. Chain of Thought with In-Context Learning
  7. Graphical Reasoning
  8. Entity Extraction
  9. Text Paraphrase
  10. Relation Extraction
  11. Evaluation
  12. Datasets
  13. ADE (Adverse Drug Event)
  14. CoNLL04
  15. NYT (New York Times)
  16. ...and 9 more sections

Figures (1)

  • Figure 1: Structure of GRE-GPT