A Compressive Memory-based Retrieval Approach for Event Argument Extraction

TL;DR

This paper tackles two core bottlenecks in retrieval-augmented event argument extraction: input length limits and a gap between retrievers and inference models. It introduces Compressive Memory-based Retrieval (CMR), a dynamic, fixed-size memory matrix that stores demonstrations and is integrated into transformer attention to enable unlimited-ish retrieval and adaptive filtering. Through pre-loading demonstrations and memory-guided retrieval, CMR achieves state-of-the-art results on RAMS, WikiEvents, and ACE2005, and demonstrates robustness and improved generalization, including application to decoder-only LLMs. However, gains for large language models are more modest, revealing the need for more diverse training data and further methodological refinements. Overall, CMR offers a scalable, memory-augmented approach to enhance retrieval quality and diversity in EAE tasks, with potential applicability to broader long-context reasoning tasks.

Abstract

Recent works have demonstrated the effectiveness of retrieval augmentation in the Event Argument Extraction (EAE) task. However, existing retrieval-based EAE methods have two main limitations: (1) input length constraints and (2) the gap between the retriever and the inference model. These issues limit the diversity and quality of the retrieved information. In this paper, we propose a Compressive Memory-based Retrieval (CMR) mechanism for EAE, which addresses the two limitations mentioned above. Our compressive memory, designed as a dynamic matrix that effectively caches retrieved information and supports continuous updates, overcomes the limitations of the input length. Additionally, after pre-loading all candidate demonstrations into the compressive memory, the model further retrieves and filters relevant information from memory based on the input query, bridging the gap between the retriever and the inference model. Extensive experiments show that our method achieves new state-of-the-art performance on three public datasets (RAMS, WikiEvents, ACE05), significantly outperforming existing retrieval-based EAE methods.

Figures (6)

• Figure 1: An example of an EAE task from the RAMS dataset ebner2020multi. Each underlined section in the template (prompt), known as a role slot, corresponds to a specific argument role.
• Figure 2: Overview of Compressive Memory-based Retrieval architecture. "CM" denotes the Compressive Memory. First, the model pre-loads all retrieved candidate demonstrations to build the memory. Then, it dynamically retrieves information from the memory based on the input query, and subsequently generates the final prediction.
• Figure 3: Demonstrations order experiment for PAIE-CMR. Normal uses the top-k demonstrations in their original retrieved order, Reverse uses them in the opposite order, and Shuffle means randomly shuffling the demonstrations.
• Figure 4: Our designed prompt for EAE task for normal decoder-only LLMs.
• Figure 5: Our designed prompt for EAE task for our CMR-based LLMs.