llmNER: (Zero|Few)-Shot Named Entity Recognition, Exploiting the Power of Large Language Models

TL;DR

llmNER tackles zero-shot and few-shot NER using large language models by encapsulating prompting, model querying, and output parsing in a Python library with a scikit-learn-like API. It provides multiple prompting strategies (single-turn, multi-turn), answer shapes (inline tags, JSON), and POS augmentation, enabling rapid prompt engineering without model fine-tuning. Evaluations on CoNLL-2003 and CoNLL-2002 show GPT-3.5 often yields strong $F_1$ scores across settings, though results are highly task- and language-dependent and not directly comparable to SOTA finetuned models. By lowering the barrier to in-context learning for NER, llmNER supports reproducibility and accelerates applied NLP research, including pre-annotation in clinical contexts.

Abstract

Large language models (LLMs) allow us to generate high-quality human-like text. One interesting task in natural language processing (NLP) is named entity recognition (NER), which seeks to detect mentions of relevant information in documents. This paper presents llmNER, a Python library for implementing zero-shot and few-shot NER with LLMs; by providing an easy-to-use interface, llmNER can compose prompts, query the model, and parse the completion returned by the LLM. Also, the library enables the user to perform prompt engineering efficiently by providing a simple interface to test multiple variables. We validated our software on two NER tasks to show the library's flexibility. llmNER aims to push the boundaries of in-context learning research by removing the barrier of the prompting and parsing steps.

Figures (3)

• Figure 1: Diagram of the architecture of the library and the pipeline for Named Entity Recognition.
• Figure 2: Example usage of llmNER for zero-shot NER.
• Figure 3: Example usage of llmNER for few-shot NER.