GLiREL -- Generalist Model for Zero-Shot Relation Extraction

TL;DR

GLiREL introduces a generalist, single-pass architecture for zero-shot relation extraction that encodes relation labels and entity-pair representations in a shared latent space and scores their compatibility via a sigmoid. The model processes multiple entity pairs and labels in parallel, enhancing efficiency over per-pair, per-label approaches, and leverages a synthetic data generation protocol to achieve strong zero-shot performance. Empirical results on Wiki-ZSL and FewRel demonstrate state-of-the-art performance, with synthetic pretraining further boosting accuracy and robustness as unseen-label counts rise. The work also provides a practical data-generation protocol and discusses extensions to coreference and document-level relation classification, highlighting GLiREL’s potential for scalable real-world IE tasks.

Abstract

We introduce GLiREL (Generalist Lightweight model for zero-shot Relation Extraction), an efficient architecture and training paradigm for zero-shot relation classification. Inspired by recent advancements in zero-shot named entity recognition, this work presents an approach to efficiently and accurately predict zero-shot relationship labels between multiple entities in a single forward pass. Experiments using the FewRel and WikiZSL benchmarks demonstrate that our approach achieves state-of-the-art results on the zero-shot relation classification task. In addition, we contribute a protocol for synthetically-generating datasets with diverse relation labels.

Figures (9)

• Figure 1: Example inputs and outputs for GLiREL.
• Figure 2: Our proposed approach to zero-shot relation extraction. Firstly, the relation labels and N entities are encoded using a bidirectional transformer. The N entity embeddings will be concatenated to form $N^2$ pairs in the relation representation layer. The relation labels are fed through a feed-forward network to obtain relation type embeddings. A scoring layer then computes the similarity between every label and every entity pair. Diagram adapted from zaratiana2023gliner.
• Figure 3: GLiREL input includes relation types $t_{0}, ..., t_{M-1}$, text tokens $x_0, ..., x_{N-1}$, and the start and end indices of all entities within the text.
• Figure 4: Addition of random drop: The effect of randomly dropping relation labels during training on the FewRel and WikiZSL datasets. Using $m=15$.
• Figure 5: Addition of refinement layers: The effect of adding refine layers for entity pair and relation labels representations. From the WikiZSL dataset, using $m=15$