Low-Resource Named Entity Recognition with Cross-Lingual, Character-Level Neural Conditional Random Fields

TL;DR

This work tackles low-resource NER by introducing a cross-lingual transfer approach that jointly trains a character-level neural CRF across related high-resource languages and the target low-resource language. The model employs a shared character encoder and language embeddings within a CRF framework, enabling cross-lingual transfer through a combined objective over target and source data. Empirical results on 15 languages reveal that neural CRFs excel in high-resource settings, while log-linear models can dominate in ultra-low-resource cases; crucially, incorporating cross-lingual supervision allows neural methods to surpass the log-linear baseline and reduce the data requirements for competitive NER performance. Overall, the findings demonstrate the viability of cross-lingual neural transfer to induce cross-language entity abstractions and reduce annotated data needs for NER in diverse languages.

Abstract

Low-resource named entity recognition is still an open problem in NLP. Most state-of-the-art systems require tens of thousands of annotated sentences in order to obtain high performance. However, for most of the world's languages, it is unfeasible to obtain such annotation. In this paper, we present a transfer learning scheme, whereby we train character-level neural CRFs to predict named entities for both high-resource languages and low resource languages jointly. Learning character representations for multiple related languages allows transfer among the languages, improving F1 by up to 9.8 points over a loglinear CRF baseline.

Figures (1)

• Figure 1: Example of an English sentence annotated with its typed named entities.