Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes

Abdullah Al Monsur, Nitesh Vamshi Bommisetty, Gene Louis Kim

TL;DR

The paper addresses the limitations of decoder-only LLMs in event detection and the reliance on Micro-F1 by proposing a context-aware, LoRA-finetuned framework that leverages sentence-level context. It introduces multiple LLM-as-Embedding variants (BaseTE, ConcatPool, FiLM, BiSE-BiLSTM, Gated Context Fusion) and contrasts prompted approaches (zero-shot and few-shot) against supervised finetuning. Key findings show that sentence-level context and LoRA substantially improve Macro-F1, particularly for long-tailed event types, with FiLM and BiSE-BiLSTM variants performing strongly across models; LoRA also acts as a regularizer and improves generalization while reducing training costs. The work demonstrates that decoder-only LLMs can achieve competitive ED performance when augmented with context integration and efficient fine-tuning, guiding practical deployment where tail-class performance is critical and prompting can be leveraged for data-scarce settings.

Abstract

The current state of event detection research has two notable re-occurring limitations that we investigate in this study. First, the unidirectional nature of decoder-only LLMs presents a fundamental architectural bottleneck for natural language understanding tasks that depend on rich, bidirectional context. Second, we confront the conventional reliance on Micro-F1 scores in event detection literature, which systematically inflates performance by favoring majority classes. Instead, we focus on Macro-F1 as a more representative measure of a model's ability across the long-tail of event types. Our experiments demonstrate that models enhanced with sentence context achieve superior performance over canonical decoder-only baselines. Using Low-Rank Adaptation (LoRA) during finetuning provides a substantial boost in Macro-F1 scores in particular, especially for the decoder-only models, showing that LoRA can be an effective tool to enhance LLMs' performance on long-tailed event classes.

Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes

TL;DR

The paper addresses the limitations of decoder-only LLMs in event detection and the reliance on Micro-F1 by proposing a context-aware, LoRA-finetuned framework that leverages sentence-level context. It introduces multiple LLM-as-Embedding variants (BaseTE, ConcatPool, FiLM, BiSE-BiLSTM, Gated Context Fusion) and contrasts prompted approaches (zero-shot and few-shot) against supervised finetuning. Key findings show that sentence-level context and LoRA substantially improve Macro-F1, particularly for long-tailed event types, with FiLM and BiSE-BiLSTM variants performing strongly across models; LoRA also acts as a regularizer and improves generalization while reducing training costs. The work demonstrates that decoder-only LLMs can achieve competitive ED performance when augmented with context integration and efficient fine-tuning, guiding practical deployment where tail-class performance is critical and prompting can be leveraged for data-scarce settings.

Abstract

The current state of event detection research has two notable re-occurring limitations that we investigate in this study. First, the unidirectional nature of decoder-only LLMs presents a fundamental architectural bottleneck for natural language understanding tasks that depend on rich, bidirectional context. Second, we confront the conventional reliance on Micro-F1 scores in event detection literature, which systematically inflates performance by favoring majority classes. Instead, we focus on Macro-F1 as a more representative measure of a model's ability across the long-tail of event types. Our experiments demonstrate that models enhanced with sentence context achieve superior performance over canonical decoder-only baselines. Using Low-Rank Adaptation (LoRA) during finetuning provides a substantial boost in Macro-F1 scores in particular, especially for the decoder-only models, showing that LoRA can be an effective tool to enhance LLMs' performance on long-tailed event classes.
Paper Structure (53 sections, 4 equations, 17 figures, 13 tables)

This paper contains 53 sections, 4 equations, 17 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Task Definition
  5. Prompted LLMs
  6. LLM-as-Embedding
  7. BaseTE: Baseline Trigger Encoder
  8. ConcatPool
  9. 1. Global Context Generation.
  10. 2. Feature Concatenation.
  11. 3. Linear Fusion.
  12. FiLM: Feature-wise Linear Modulation
  13. 1. Global Context Generation.
  14. 2. FiLM Parameter Generation.
  15. Feature-wise Modulation.
  16. ...and 38 more sections

Figures (17)

  • Figure 1: Macro-F1 scores across quartiles of event types, ordered by event mention frequency for four models, BERT, T5, Llama 1B FiLM, and Qwen 3B FiLM. We find that models consistently underperform on the events with fewest mentions (left side of the plot). The top of the diagram shows examples of high-frequency event, Warning, with a correct prediction and low-frequency event, Incident, with an incorrect prediction from Qwen 3B FiLM.
  • Figure 2: Distribution of event types by event mention instances in the MAVEN and RAMS dataset.
  • Figure 3: Few-shot Prompt: Uses example-based contextual learning with explicit JSON formatting and multi-trigger handling.
  • Figure 4: Zero-shot Prompt: Relies purely on explicit instruction-based reasoning with structured JSON output.
  • Figure 5: Overview of the Event Detection Framework
  • ...and 12 more figures