The Million-Label NER: Breaking Scale Barriers with GLiNER bi-encoder

TL;DR

GLiNER-bi-Encoder introduces a decoupled bi-encoder architecture for NER that eliminates the context-window bottleneck by pre-computing label embeddings and scoring text spans against a label space. It achieves state-of-the-art zero-shot performance on CrossNER ($\$61.5\%$ Micro-F1) and delivers up to $\$130\times$ faster inference at $1024$ labels using pre-computed label embeddings, while maintaining competitive accuracy with uni-encoder baselines. The work also introduces GLiNKER, a modular entity-linking framework that leverages the bi-encoder for scalable disambiguation across massive knowledge bases. Across 26 datasets, bi-encoder models match or exceed uni-encoder performance, with pronounced gains in biomedical and ontology-rich domains, and demonstrate practical deployment benefits through near-constant inference costs when label embeddings are cached. Limitations include highly contextual or long-span entities and a fixed maximum span width, pointing to future directions in contrastive pre-training, retrieval-augmented learning, and multilingual expansion.

Abstract

This paper introduces GLiNER-bi-Encoder, a novel architecture for Named Entity Recognition (NER) that harmonizes zero-shot flexibility with industrial-scale efficiency. While the original GLiNER framework offers strong generalization, its joint-encoding approach suffers from quadratic complexity as the number of entity labels increases. Our proposed bi-encoder design decouples the process into a dedicated label encoder and a context encoder, effectively removing the context-window bottleneck. This architecture enables the simultaneous recognition of thousands, and potentially millions, of entity types with minimal overhead. Experimental results demonstrate state-of-the-art zero-shot performance, achieving 61.5 percent Micro-F1 on the CrossNER benchmark. Crucially, by leveraging pre-computed label embeddings, GLiNER-bi-Encoder achieves up to a 130 times throughput improvement at 1024 labels compared to its uni-encoder predecessors. Furthermore, we introduce GLiNKER, a modular framework that leverages this architecture for high-performance entity linking across massive knowledge bases such as Wikidata.

Figures (1)

• Figure 1: GLiNER architecture variants. Left: Uni-encoder processes entity types and text jointly. Right: Bi-encoder uses separate encoders with optional cross-attention fusion.