Large-Scale Label Interpretation Learning for Few-Shot Named Entity Recognition

TL;DR

This work introduces LitSet, a scalable label interpretation learning approach that leverages ZELDA and WikiData to massively expand the space and granularity of entity-type descriptions used in few-shot NER. By training a bi-encoder on a large, richly described label set and restricting the loss to labels in the current batch, LitSet achieves strong zero- and few-shot NER performance across in-domain, cross-domain, and cross-lingual scenarios, outperforming several baselines and transferring across architectures. The authors provide extensive validation showing that both the number of distinct labels and the expressiveness of label descriptions systematically improve performance, and they release the LitSet dataset and code for public use. While promising, the approach faces challenges from annotation noise and language transfer, suggesting future work to broaden benchmarks and refine labeling pipelines for even better robustness and zero-shot capabilities.

Abstract

Few-shot named entity recognition (NER) detects named entities within text using only a few annotated examples. One promising line of research is to leverage natural language descriptions of each entity type: the common label PER might, for example, be verbalized as ''person entity.'' In an initial label interpretation learning phase, the model learns to interpret such verbalized descriptions of entity types. In a subsequent few-shot tagset extension phase, this model is then given a description of a previously unseen entity type (such as ''music album'') and optionally a few training examples to perform few-shot NER for this type. In this paper, we systematically explore the impact of a strong semantic prior to interpret verbalizations of new entity types by massively scaling up the number and granularity of entity types used for label interpretation learning. To this end, we leverage an entity linking benchmark to create a dataset with orders of magnitude of more distinct entity types and descriptions as currently used datasets. We find that this increased signal yields strong results in zero- and few-shot NER in in-domain, cross-domain, and even cross-lingual settings. Our findings indicate significant potential for improving few-shot NER through heuristical data-based optimization.

Figures (5)

• Figure 1: Given existing datasets, few-shot NER methods requiring an initial label interpretation learning are limited regarding entity types and label verbalizations. We propose learning from orders of magnitude more distinct types and more expressive label semantics than current datasets by utilizing ZELDA annotated with WikiData information.
• Figure 2: F1 scores for few-shot NER tagset extension on FewNERD depending on how many distinct entity types were seen in label interpretation learning (columns) and how label types were verbalized (rows). We report F1 scores averaged over five seeds. We observe that (1) more distinct labels during label interpretation training and (2) more semantically expressive labels improve the few-shot ability on unseen labels.
• Figure 3: An example annotation of a sentence in ZELDA. WikiData provides precise descriptions and labels about an entity. Annotation types in existing datasets (CoNLL-03, FewNERD) are be less informative if not misleading.
• Figure 4: Exemplary illustration on the INTRA and INTER settings of FewNERD experiments.
• Figure 5: $K$-shot tagset extension on the 16 least occurring labels of FewNERD using the sparse-latent-typing encoder. We sweep over different numbers of distinct entity types and different semantic descriptions observed during label interpretation learning. We find that increasing both dimensions (more distinct types, extensive label verbalizations) contributes to an improved few-shot generalization.