ReverseNER: A Self-Generated Example-Driven Framework for Zero-Shot Named Entity Recognition with Large Language Models

TL;DR

ReverseNER tackles zero-shot NER by reversing the traditional labeling process to generate a task-specific, high-quality example library from entity definitions. It combines feature sentence extraction via BERT embeddings and $k$-Medoids clustering, a reverse vocabulary expansion to create labeled examples, and an example-driven inference stage with entity-level self-consistency scoring to boost precision. Across English and Chinese benchmarks, ReverseNER improves micro F1 over zero-shot baselines and conventional LLM-based approaches while reducing overall LLM invocations, with SC further enhancing performance in many settings. The approach enables reliable, low-resource zero-shot NER and suggests a broader potential for self-hinted, reverse-proc framework applications in related NLP tasks.

Abstract

This paper presents ReverseNER, a method aimed at overcoming the limitation of large language models (LLMs) in zero-shot named entity recognition (NER) tasks, arising from their reliance on pre-provided demonstrations. ReverseNER tackles this challenge by constructing a reliable example library composed of dozens of entity-labeled sentences, generated through the reverse process of NER. Specifically, while conventional NER methods label entities in a sentence, ReverseNER features reversing the process by using an LLM to generate entities from their definitions and subsequently expand them into full sentences. During the entity expansion process, the LLM is guided to generate sentences by replicating the structures of a set of specific \textsl{feature sentences}, extracted from the task sentences by clustering. This expansion process produces dozens of entity-labeled task-relevant sentences. After constructing the example library, the method selects several semantically similar entity-labeled examples for each task sentence as references to facilitate the LLM's entity recognition. We also propose an entity-level self-consistency scoring mechanism to improve NER performance with LLMs. Experiments show that ReverseNER significantly outperforms other zero-shot NER methods with LLMs, marking a notable improvement in NER for domains without labeled data, while declining computational resource consumption.

Figures (3)

• Figure 1: The processes of NER and Reversed NER. Both of them output entity-labeled sentences. (a) NER Process takes raw texts as input and outputs the result through entity labeling; (b) Our Reversed NER Process takes definitions of entities as the only input and outputs the result through generating entities and expanding entities into sentences. The Reversed NER Process requires a generative LLM to work.
• Figure 2: Overview of the entire ReverseNER Process: Clustering, Constructing Entity-labeled Exapmles and Selecting .
• Figure 3: Average micro F1 on WikiGold when the number of clusters is fixed at 10, evaluated on GPT-4o mini.