Small Models Are (Still) Effective Cross-Domain Argument Extractors

TL;DR

The paper addresses cross-domain transfer in event argument extraction (EAE) by recasting into QA and TI. It demonstrates that small Flan-T5 models trained on a suitable source ontology can achieve zero-shot $F_1$ performance competitive with or surpassing GPT-3.5 and GPT-4 across six datasets. It provides expert-written questions and templates for all six ontologies and analyzes transfer dynamics, including in-domain vs. out-of-domain performance and paraphrase augmentation. The results argue for using compact, ontology-tuned models as a practical first line for cross-domain EAE and highlight continued challenges for highly distant ontology transfer.

Abstract

Effective ontology transfer has been a major goal of recent work on event argument extraction (EAE). Two methods in particular -- question answering (QA) and template infilling (TI) -- have emerged as promising approaches to this problem. However, detailed explorations of these techniques' ability to actually enable this transfer are lacking. In this work, we provide such a study, exploring zero-shot transfer using both techniques on six major EAE datasets at both the sentence and document levels. Further, we challenge the growing reliance on LLMs for zero-shot extraction, showing that vastly smaller models trained on an appropriate source ontology can yield zero-shot performance superior to that of GPT-3.5 or GPT-4.

Figures (3)

• Figure 1: Example event types from WikiEvents (left) and from FAMuS (right), along with the templates and questions used for them in this work.
• Figure 2: Pearson's $\rho$ between per-event type in-domain and zero-shot argument $\text{F}_1$.
• Figure 3: Argument $\text{F}_1$ of QA and TI models trained on FAMuS with and without an additional five paraphrases per question/template.