GLiNER multi-task: Generalist Lightweight Model for Various Information Extraction Tasks

TL;DR

IE models must balance accuracy, efficiency, and adaptability across diverse tasks. GLiNER multi-task leverages an encoder-based, token-classification approach with joint token-label representations, enabling structured outputs and longer sequence extraction. It combines synthetic data generation via Llama3 8B, a two-stage fine-tuning regime, and self-learning to achieve strong cross-task performance, including SoTA zero-shot NER and top QA and summarization results, often outperforming teacher models on several tasks. The approach demonstrates that compact encoders, when trained on diverse data and augmented with self-learning, can deliver scalable, controllable, and generalizable information extraction across multiple tasks with practical deployment benefits.

Abstract

Information extraction tasks require both accurate, efficient, and generalisable models. Classical supervised deep learning approaches can achieve the required performance, but they need large datasets and are limited in their ability to adapt to different tasks. On the other hand, large language models (LLMs) demonstrate good generalization, meaning that they can adapt to many different tasks based on user requests. However, LLMs are computationally expensive and tend to fail to generate structured outputs. In this article, we will introduce a new kind of GLiNER model that can be used for various information extraction tasks while being a small encoder model. Our model achieved SoTA performance on zero-shot NER benchmarks and leading performance on question-answering, summarization and relation extraction tasks. Additionally, in this article, we will cover experimental results on self-learning approaches for named entity recognition using GLiNER models.

Figures (1)

• Figure 1: F1 scores for Named Entity Recognition tasks across different models and datasets.