WikiNER-fr-gold: A Gold-Standard NER Corpus
Danrun Cao, Nicolas Béchet, Pierre-François Marteau
TL;DR
This paper evaluates the quality of WikiNER and delivers WikiNER-fr-gold, a gold-standard revision of the French portion created by manually revising 20% of WikiNER-fr (26,818 sentences, ~700k tokens) and establishing a standardized annotation guideline. It documents the original WikiNER production pipeline, where Wikipedia hyperlinks and a French classifier (trained on ~2,500 articles with a high F1 score) were used to generate semi-supervised annotations and multiple variants, from which WIKI-2 level-2 was selected. The authors define a comprehensive French NER taxonomy, specify BIOES tagging with 17 labels, and describe the annotation tool used. They perform error analysis, focusing on hyperlink inconsistencies, non-conforming links, and complex entity cases, and propose correction rules and future work involving automation, active learning, and expansion to the full corpus and other languages.
Abstract
We address in this article the the quality of the WikiNER corpus, a multilingual Named Entity Recognition corpus, and provide a consolidated version of it. The annotation of WikiNER was produced in a semi-supervised manner i.e. no manual verification has been carried out a posteriori. Such corpus is called silver-standard. In this paper we propose WikiNER-fr-gold which is a revised version of the French proportion of WikiNER. Our corpus consists of randomly sampled 20% of the original French sub-corpus (26,818 sentences with 700k tokens). We start by summarizing the entity types included in each category in order to define an annotation guideline, and then we proceed to revise the corpus. Finally we present an analysis of errors and inconsistency observed in the WikiNER-fr corpus, and we discuss potential future work directions.