Wiki-Quantities and Wiki-Measurements: Datasets of Quantities and their Measurement Context from Wikipedia

TL;DR

Two large datasets based on Wikipedia and Wikidata, consisting of over 1.2 million annotated quantities in the English-language Wikipedia and Wiki-Measurements, are presented and used in pipeline approaches to measurement extraction.

Abstract

To cope with the large number of publications, more and more researchers are automatically extracting data of interest using natural language processing methods based on supervised learning. Much data, especially in the natural and engineering sciences, is quantitative, but there is a lack of datasets for identifying quantities and their context in text. To address this issue, we present two large datasets based on Wikipedia and Wikidata: Wiki-Quantities is a dataset consisting of over 1.2 million annotated quantities in the English-language Wikipedia. Wiki-Measurements is a dataset of 38,738 annotated quantities in the English-language Wikipedia along with their respective measured entity, property, and optional qualifiers. Manual validation of 100 samples each of Wiki-Quantities and Wiki-Measurements found 100% and 84-94% correct, respectively. The datasets can be used in pipeline approaches to measurement extraction, where quantities are first identified and then their measurement context. To allow reproduction of this work using newer or different versions of Wikipedia and Wikidata, we publish the code used to create the datasets along with the data.

Figures (1)

• Figure 1: Schematic illustration of the methodology used to create the datasets: a) Wiki-Quantities uses convert template calls as quantity annotations. b) Wiki-Measurements aligns Wikidata facts with text from their respective Wikipedia page. Both implicit and explicit properties are supported. The text examples are from Wikipedia.