CodeNER: Code Prompting for Named Entity Recognition

TL;DR

This work tackles zero-shot and few-shot NER with large language models by bridging the gap between text-in-text-out prompting and the text-in-span-out BIO labeling requirement. It introduces CodeNER, a code-based prompting framework that embeds explicit BIO schema and input sentence context within a programming-language-style prompt to guide LLMs in identifying accurate entity boundaries. Across ten multilingual benchmarks and multiple model families, CodeNER consistently outperforms traditional text-based prompts, with additional gains when combined with chain-of-thought reasoning. The study demonstrates the potential of structured, code-based prompts for complex sequence labeling tasks and outlines practical considerations and limitations for future refinement.

Abstract

Recent studies have explored various approaches for treating candidate named entity spans as both source and target sequences in named entity recognition (NER) by leveraging large language models (LLMs). Although previous approaches have successfully generated candidate named entity spans with suitable labels, they rely solely on input context information when using LLMs, particularly, ChatGPT. However, NER inherently requires capturing detailed labeling requirements with input context information. To address this issue, we propose a novel method that leverages code-based prompting to improve the capabilities of LLMs in understanding and performing NER. By embedding code within prompts, we provide detailed BIO schema instructions for labeling, thereby exploiting the ability of LLMs to comprehend long-range scopes in programming languages. Experimental results demonstrate that the proposed code-based prompting method outperforms conventional text-based prompting on ten benchmarks across English, Arabic, Finnish, Danish, and German datasets, indicating the effectiveness of explicitly structuring NER instructions. We also verify that combining the proposed code-based prompting method with the chain-of-thought prompting further improves performance.

Figures (7)

• Figure 1: Examples of text- and code-based prompts for zero-shot NER with ChatGPT. (a) and (b) indicate the text-based and code-based prompts, respectively.
• Figure 2: Experimental results for each label on MIT-Restaurant and MIT-Movie.
• Figure 3: Case studies of Vanilla and CodeNER outputs. Text denotes the input sentence, and Gold refers to the correct label for Text. Vanilla Output denotes the results generated by Vanilla. Vanilla Result represents the conversion of Vanilla Output into the BIO-tagging format for evaluation. CodeNER Result denotes the CodeNER labeling results for Text.
• Figure 4: Examples of prompts for Vanilla with label descriptions. (a) is the examples of Vanilla with label same as codeNER and (b) is the examples of Vanilla with label generated by GPT.
• Figure 5: GoLLIE prompt for CoNLL03, which includes labels such as PER, ORG, LOC, and MISC.