Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models

TL;DR

The first to reformulate the document-level relation extraction task as a tailored in-context few-shot learning paradigm is presented, achieving state-of-the-art results across six relation extraction datasets and outperforming more than 30 baseline methods.

Abstract

Document-level relation extraction aims at inferring structured human knowledge from textual documents. State-of-the-art methods for this task use pre-trained language models (LMs) via fine-tuning, yet fine-tuning is computationally expensive and cannot adapt to new relation types or new LMs. As a remedy, we leverage the generalization capabilities of pre-trained LMs and present a novel framework for document-level in-context few-shot relation extraction. Our framework has three strengths: it eliminates the need (1) for named entity recognition and (2) for human annotations of documents, and (3) it can be updated to new LMs without re-training. We evaluate our framework using DocRED, the largest publicly available dataset for document-level relation extraction, and demonstrate that our framework achieves state-of-the-art performance. We further show that our framework actually performs much better than the original labels from the development set of DocRED. Finally, we conduct an extensive benchmark demonstrating the effectiveness of our framework, achieving state-of-the-art results across six relation extraction datasets and outperforming more than 30 baseline methods. Unlike our framework, the baseline methods have large computational overhead (e.g., from fine-tuning). To the best of our knowledge, we are the first to reformulate the document-level relation extraction task as a tailored in-context few-shot learning paradigm.

Figures (6)

• Figure 1: Overview of our REPLM. Our framework takes a new document and relation as input and then proceeds along three steps: (1) selects a candidate pool of $N$ in-context examples; (2) constructs $L$ sets of such in-context examples; and (3) calculates the joint probabilities of subject-object pairs to extract knowledge triplets. Legend: subjects and objects are colored in blue and orange, respectively.
• Figure 2: F1 scores per relation type (darker = better). Missing color means that no correct predictions were made for this relation. F1 scores are normalized by the maximum value for each relation. Relations are in decreasing order of their number of knowledge triplets.
• Figure 3: Ablation studies on CONLL04.
• Figure 4: Histogram of average cosine similarity between documents and their top-$N$ neighbors for two example relation types.
• Figure 5: Comparing the performance of our REPLM against GPT-RE wan2023gpt on CONLL04 dataset.