Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Document-level Argument Extraction with Definition-augmented Heuristic-driven Prompting for LLMs

Tongyue Sun, Jiayi Xiao

TL;DR

The paper tackles document-level Event Argument Extraction (EAE), a task hampered by data scarcity and complex cross-sentential reasoning. It introduces Definition-augmented Heuristic-driven Prompting (DHP), which integrates explicit event definitions, extraction rules, and heuristic-driven Chain-of-Thought with optimized prompt length to guide LLMs. Empirical results on RAMS and DocEE show that DHP improves Arg-I and Arg-C F1 scores over CoT prompting and several supervised baselines, with notable gains in cross-domain settings, indicating better generalization and reduced labeling needs. The method advances practical document-level EAE by combining structured definitions, rule-based heuristics, and reasoning prompts to improve accuracy and interpretability in LLM-driven extraction.

Abstract

Event Argument Extraction (EAE) is pivotal for extracting structured information from unstructured text, yet it remains challenging due to the complexity of real-world document-level EAE. We propose a novel Definition-augmented Heuristic-driven Prompting (DHP) method to enhance the performance of Large Language Models (LLMs) in document-level EAE. Our method integrates argument extraction-related definitions and heuristic rules to guide the extraction process, reducing error propagation and improving task accuracy. We also employ the Chain-of-Thought (CoT) method to simulate human reasoning, breaking down complex problems into manageable sub-problems. Experiments have shown that our method achieves a certain improvement in performance over existing prompting methods and few-shot supervised learning on document-level EAE datasets. The DHP method enhances the generalization capability of LLMs and reduces reliance on large annotated datasets, offering a novel research perspective for document-level EAE.

Enhancing Document-level Argument Extraction with Definition-augmented Heuristic-driven Prompting for LLMs

TL;DR

The paper tackles document-level Event Argument Extraction (EAE), a task hampered by data scarcity and complex cross-sentential reasoning. It introduces Definition-augmented Heuristic-driven Prompting (DHP), which integrates explicit event definitions, extraction rules, and heuristic-driven Chain-of-Thought with optimized prompt length to guide LLMs. Empirical results on RAMS and DocEE show that DHP improves Arg-I and Arg-C F1 scores over CoT prompting and several supervised baselines, with notable gains in cross-domain settings, indicating better generalization and reduced labeling needs. The method advances practical document-level EAE by combining structured definitions, rule-based heuristics, and reasoning prompts to improve accuracy and interpretability in LLM-driven extraction.

Abstract

Event Argument Extraction (EAE) is pivotal for extracting structured information from unstructured text, yet it remains challenging due to the complexity of real-world document-level EAE. We propose a novel Definition-augmented Heuristic-driven Prompting (DHP) method to enhance the performance of Large Language Models (LLMs) in document-level EAE. Our method integrates argument extraction-related definitions and heuristic rules to guide the extraction process, reducing error propagation and improving task accuracy. We also employ the Chain-of-Thought (CoT) method to simulate human reasoning, breaking down complex problems into manageable sub-problems. Experiments have shown that our method achieves a certain improvement in performance over existing prompting methods and few-shot supervised learning on document-level EAE datasets. The DHP method enhances the generalization capability of LLMs and reduces reliance on large annotated datasets, offering a novel research perspective for document-level EAE.
Paper Structure (11 sections, 1 figure, 3 tables)

This paper contains 11 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Approach
  3. Experiments
  4. Setup
  5. Results
  6. Analysis
  7. Related works
  8. Document-level EAE
  9. In-Context Learning
  10. Conclusion
  11. Experimental Details

Figures (1)

  • Figure 1: Definition-augmented Heuristic-driven Prompting method guides on how to extract event arguments related to specific trigger words from documents by defining the task, terminology, extraction rules, and a list of heuristics. It provides corresponding definitions for argument extraction prompting and heuristic rules to assist in the identification and extraction of event arguments.