MAQInstruct: Instruction-based Unified Event Relation Extraction

TL;DR

MAQInstruct tackles the challenge of instruction-based event relation extraction by (1) reframing instructions to select event mentions via event-relations, reducing inference samples from $n \times n$ to $k \times n$, and (2) introducing a bipartite matching loss to decouple output order from relation extraction. The approach yields significant gains over prior instruction-based methods across multiple LLMs and datasets, with improved sample efficiency and robustness to instruction design. It also demonstrates strong zero-shot capabilities and informative ablations, highlighting the importance of dependency parsing chains and matching losses. The work advances practical ERE by enabling unified, instruction-driven extraction without relying on predefined relation schemas, with potential impact on real-world information extraction pipelines.

Abstract

Extracting event relations that deviate from known schemas has proven challenging for previous methods based on multi-class classification, MASK prediction, or prototype matching. Recent advancements in large language models have shown impressive performance through instruction tuning. Nevertheless, in the task of event relation extraction, instruction-based methods face several challenges: there are a vast number of inference samples, and the relations between events are non-sequential. To tackle these challenges, we present an improved instruction-based event relation extraction framework named MAQInstruct. Firstly, we transform the task from extracting event relations using given event-event instructions to selecting events using given event-relation instructions, which reduces the number of samples required for inference. Then, by incorporating a bipartite matching loss, we reduce the dependency of the instruction-based method on the generation sequence. Our experimental results demonstrate that MAQInstruct significantly improves the performance of event relation extraction across multiple LLMs.

Figures (7)

• Figure 1: Different event relation extraction methods.
• Figure 2: Cross-entropy loss vs bipartite matching loss.
• Figure 3: The performance of different answer sequences.
• Figure 4: The F1 score of different instruction-based LLMs on the zero-shot event relation extraction task.
• Figure 5: The performance of different LLMs on natural language understanding evaluation datasets.