GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer

TL;DR

GLiNER tackles open-type NER with a compact bidirectional LM that matches entity-type embeddings to span representations, enabling parallel extraction and efficient inference. By training on the diverse Pile-NER corpus and using a span-based decoding strategy, GLiNER achieves competitive to superior zero-shot performance compared to ChatGPT and fine-tuned LLMs, across English and multilingual datasets, while maintaining much smaller compute requirements. Key contributions include a unified latent-space formulation for entity-type and span representations, a scalable O(n log n) decoding scheme, and extensive ablations showing robustness across backbones and data regimes. This approach offers a practical, resource-efficient alternative for open-type NER with strong cross-domain and cross-language generalization, suitable for deployment in compute-constrained settings.

Abstract

Named Entity Recognition (NER) is essential in various Natural Language Processing (NLP) applications. Traditional NER models are effective but limited to a set of predefined entity types. In contrast, Large Language Models (LLMs) can extract arbitrary entities through natural language instructions, offering greater flexibility. However, their size and cost, particularly for those accessed via APIs like ChatGPT, make them impractical in resource-limited scenarios. In this paper, we introduce a compact NER model trained to identify any type of entity. Leveraging a bidirectional transformer encoder, our model, GLiNER, facilitates parallel entity extraction, an advantage over the slow sequential token generation of LLMs. Through comprehensive testing, GLiNER demonstrate strong performance, outperforming both ChatGPT and fine-tuned LLMs in zero-shot evaluations on various NER benchmarks.

Figures (7)

• Figure 1: BiLM for Open NER. Previous models, such as UniNER zhou2023universalner (Fig. a), approach the task of open type NER by prompting LLMs with natural language instructions (using a multi-turn dialogue style). Our proposed GLiNER utilizes small bidirectional LMs and treats the NER task as matching entity types with textual spans in a latent space.
• Figure 2: Model architecture. GLiNER employs a BiLM and takes as input entity type prompts and a sentence/text. Each entity is separated by a learned token [ENT]. The BiLM outputs representations for each token. Entity embeddings are passed into a FeedForward Network, while input word representations are passed into a span representation layer to compute embeddings for each span. Finally, we compute a matching score between entity representations and span representations (using dot product and sigmoid activation). For instance, in the figure, the span representation of (0, 1), corresponding to "Alain Farley," has a high matching score with the entity embeddings of "Person".
• Figure 3: Prompting ChatGPT for entity extraction. This prompt was used zhou2023universalner to construct the Pile-NER dataset.
• Figure 4: Zero-shot performance for different backbones. It reports the avg. results on 20 NER and OOD NER datasets
• Figure 5: Supervised performance across different dataset sizes. The evaluation is conducted on the 20 NER datasets (in table \ref{['tab:supexp']}).