Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Label Alignment and Reassignment with Generalist Large Language Model for Enhanced Cross-Domain Named Entity Recognition

Ke Bao, Chonghuan Yang

TL;DR

This work tackles cross-domain named entity recognition by addressing label conflict through a label alignment and reassignment (LAR) framework. It trains a generative LM on a rich source domain, learns a mapping from target to source entity types, and refines target-type knowledge via label reassignment, with further enhancement via GPT-3.5 for zero-shot inference. Comprehensive experiments on supervised OOD, zero-shot OOD, and supervised in-domain settings demonstrate competitive or state-of-the-art performance and highlight LAR's effectiveness in mitigating label conflicts and boosting cross-domain generalization. The approach offers practical impact for deploying NER systems across diverse domains with limited labeled data, leveraging external LLMs for disambiguation.

Abstract

Named entity recognition on the in-domain supervised and few-shot settings have been extensively discussed in the NLP community and made significant progress. However, cross-domain NER, a more common task in practical scenarios, still poses a challenge for most NER methods. Previous research efforts in that area primarily focus on knowledge transfer such as correlate label information from source to target domains but few works pay attention to the problem of label conflict. In this study, we introduce a label alignment and reassignment approach, namely LAR, to address this issue for enhanced cross-domain named entity recognition, which includes two core procedures: label alignment between source and target domains and label reassignment for type inference. The process of label reassignment can significantly be enhanced by integrating with an advanced large-scale language model such as ChatGPT. We conduct an extensive range of experiments on NER datasets involving both supervised and zero-shot scenarios. Empirical experimental results demonstrate the validation of our method with remarkable performance under the supervised and zero-shot out-of-domain settings compared to SOTA methods.

Label Alignment and Reassignment with Generalist Large Language Model for Enhanced Cross-Domain Named Entity Recognition

TL;DR

This work tackles cross-domain named entity recognition by addressing label conflict through a label alignment and reassignment (LAR) framework. It trains a generative LM on a rich source domain, learns a mapping from target to source entity types, and refines target-type knowledge via label reassignment, with further enhancement via GPT-3.5 for zero-shot inference. Comprehensive experiments on supervised OOD, zero-shot OOD, and supervised in-domain settings demonstrate competitive or state-of-the-art performance and highlight LAR's effectiveness in mitigating label conflicts and boosting cross-domain generalization. The approach offers practical impact for deploying NER systems across diverse domains with limited labeled data, leveraging external LLMs for disambiguation.

Abstract

Named entity recognition on the in-domain supervised and few-shot settings have been extensively discussed in the NLP community and made significant progress. However, cross-domain NER, a more common task in practical scenarios, still poses a challenge for most NER methods. Previous research efforts in that area primarily focus on knowledge transfer such as correlate label information from source to target domains but few works pay attention to the problem of label conflict. In this study, we introduce a label alignment and reassignment approach, namely LAR, to address this issue for enhanced cross-domain named entity recognition, which includes two core procedures: label alignment between source and target domains and label reassignment for type inference. The process of label reassignment can significantly be enhanced by integrating with an advanced large-scale language model such as ChatGPT. We conduct an extensive range of experiments on NER datasets involving both supervised and zero-shot scenarios. Empirical experimental results demonstrate the validation of our method with remarkable performance under the supervised and zero-shot out-of-domain settings compared to SOTA methods.
Paper Structure (21 sections, 4 figures, 6 tables)

This paper contains 21 sections, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Named Entity Recognition
  4. Cross-Domain Named Entity Recognition
  5. Using LLM on Named Entity Recognition
  6. Methodology
  7. Preliminary
  8. Label Alignment and Reassignment
  9. Enhanced Inference with LLM
  10. Experiment
  11. Experimental Settings
  12. Baselines
  13. Results on supervised OOD setting
  14. Results on zero-shot OOD setting
  15. Results on supervised in-domain setting
  16. ...and 6 more sections

Figures (4)

  • Figure 1: Due to the label conflict problem, the training without label alignment encounters conflicting knowledge among entities with similar context semantics on both the source and target domain datasets, while training with aligned labels can avoid this issue.
  • Figure 2: The workflow of our method, which includes all the steps from training to prediction. The solid arrows represent training processes, and the dotted arrows represent inference processes.
  • Figure 3: The trend of performance degradation of our model when it is applied to continuous learning.
  • Figure 4: The successful and failure extraction cases of our method with and without label alignment. The instances that are predicted incorrectly are marked in red and the missing instances are marked in orange. The samples and ground labels originate from the test set of the Politics dataset.