Rethinking Negative Instances for Generative Named Entity Recognition

TL;DR

Rethinking Negative Instances for Generative Named Entity Recognition investigates whether incorporating negative (non-entity) text during training can improve generative NER. The authors introduce GNER, which combines negative-instance training with an LCS-based decoding module and an instruction-tuning regimen to handle token-level labeling in a text-to-text model. They show that negative context enhances precision and recall, reduces unlabeled, noisy, and boundary errors, and, with an efficient LCS matcher, yields robust structured outputs. Across Flan-T5 and LLaMA backbones, GNER achieves strong zero-shot results, surpassing the state-of-the-art by about 9 $F_1$ points, and also demonstrates competitive supervised performance with improved efficiency. These findings highlight the value of negative instances and sequence-structure-aware decoding for cross-domain NER with generative models.

Abstract

Large Language Models (LLMs) have demonstrated impressive capabilities for generalizing in unseen tasks. In the Named Entity Recognition (NER) task, recent advancements have seen the remarkable improvement of LLMs in a broad range of entity domains via instruction tuning, by adopting entity-centric schema. In this work, we explore the potential enhancement of the existing methods by incorporating negative instances into training. Our experiments reveal that negative instances contribute to remarkable improvements by (1) introducing contextual information, and (2) clearly delineating label boundaries. Furthermore, we introduce an efficient longest common subsequence (LCS) matching algorithm, which is tailored to transform unstructured predictions into structured entities. By integrating these components, we present GNER, a Generative NER system that shows improved zero-shot performance across unseen entity domains. Our comprehensive evaluation illustrates our system's superiority, surpassing state-of-the-art (SoTA) methods by 9 $F_1$ score in zero-shot evaluation.

Figures (7)

• Figure 1: Zero-shot performance of our models. Our models GNER-LLaMA and GNER-T5 both outperform the SoTA sainz2023gollie in zero-shot settings. GPT results are from zhou2023universalner.
• Figure 2: A simplified example of instructions in InstructUIE wang2023instructuie, UniversalNER zhou2023universalner and GoLLIE sainz2023gollie.
• Figure 3: Constructed prompts used for training.
• Figure 4: Zero-shot performance of training with entity context and enhanced boundary strategies. A contextual length of 0 indicates no context is included, while a length of $N$ signifies that the entire sentence is included.
• Figure 5: Prompt used for instruction tuning.