LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition

TL;DR

This work introduces LLM-DA, a data augmentation framework for few-shot NER that leverages large language models to generate semantically coherent sentences. It augments data at both the context and entity levels using 14 context-level strategies, same-type entity substitutions, and noise injection, with a two-stage both-level approach to maximize diversity while maintaining label integrity. A greedy k-shot sampling method with a relaxed 1.25k cap supports realistic few-shot distributions, and a filtering-based data-annotation step preserves label accuracy. Empirical results across four diverse NER datasets show substantial improvements over baselines, with LLM-DA outperforming ChatGPT in most settings and yielding higher-quality augmented data in terms of informativeness, readability, and grammaticality. The findings highlight the effectiveness of combining structured prompts, world knowledge, and controlled diversity to advance few-shot NER and inform future data-centered augmentation strategies.

Abstract

Despite the impressive capabilities of large language models (LLMs), their performance on information extraction tasks is still not entirely satisfactory. However, their remarkable rewriting capabilities and extensive world knowledge offer valuable insights to improve these tasks. In this paper, we propose $LLM-DA$, a novel data augmentation technique based on LLMs for the few-shot NER task. To overcome the limitations of existing data augmentation methods that compromise semantic integrity and address the uncertainty inherent in LLM-generated text, we leverage the distinctive characteristics of the NER task by augmenting the original data at both the contextual and entity levels. Our approach involves employing 14 contextual rewriting strategies, designing entity replacements of the same type, and incorporating noise injection to enhance robustness. Extensive experiments demonstrate the effectiveness of our approach in enhancing NER model performance with limited data. Furthermore, additional analyses provide further evidence supporting the assertion that the quality of the data we generate surpasses that of other existing methods.

Figures (5)

• Figure 1: Examples of various data augmentation methods applied to the same original sentence. Entities ("event") within the sentences are bolded, while any ungrammatical or semantically incoherent parts are highlighted in red.
• Figure 2: An illustration of LLM-DA. LLM-DA comprises three stages: data sampling from the original dataset, generation of augmented data across four dimensions using LLMs with specially designed prompts, and finally, filtering the augmented data and data annotation to obtain the final valid dataset.
• Figure 3: Performance of ChatGPT using 0/1/3-sample with that of smaller model trained using LLM-DA data.
• Figure 4: The entity distribution of data augmented by DAGA (\ref{['fig:daga-conll']}) and LLM-DA (All) (\ref{['fig:ours-conll']}).
• Figure 5: Performance of BERT on the OntoNotes 5.0 (\ref{['fig:onto']}) and FEW-NERD (\ref{['fig:few-nerd']}) datasets when trained with LLM-DA (All) data at various augmentation ratios.