Curating Grounded Synthetic Data with Global Perspectives for Equitable AI

TL;DR

This work introduces a grounded synthetic data pipeline that leverages a broad, multilingual news corpus from 125 countries and 12 languages to create the AskNews-NER-v0 dataset for robust NER training. By enforcing topic diversification and using translation/summarization plus embedding-based clustering, the authors generate 5,049 summarized articles across 73 topics and 54 entity types, with splits designed for robust evaluation. Fine-tuning GLiNER variants on this dataset yields consistent gains on 18 standard NER benchmarks (average ~4.6%, up to 7.3% for larger models) and competitive OOD performance, demonstrating the value of globally representative data for generalization. The authors release the dataset and open-source models, providing a scalable blueprint for equitable AI through diversified data synthesis applicable beyond NER.

Abstract

The development of robust AI models relies heavily on the quality and variety of training data available. In fields where data scarcity is prevalent, synthetic data generation offers a vital solution. In this paper, we introduce a novel approach to creating synthetic datasets, grounded in real-world diversity and enriched through strategic diversification. We synthesize data using a comprehensive collection of news articles spanning 12 languages and originating from 125 countries, to ensure a breadth of linguistic and cultural representations. Through enforced topic diversification, translation, and summarization, the resulting dataset accurately mirrors real-world complexities and addresses the issue of underrepresentation in traditional datasets. This methodology, applied initially to Named Entity Recognition (NER), serves as a model for numerous AI disciplines where data diversification is critical for generalizability. Preliminary results demonstrate substantial improvements in performance on traditional NER benchmarks, by up to 7.3%, highlighting the effectiveness of our synthetic data in mimicking the rich, varied nuances of global data sources. This paper outlines the strategies employed for synthesizing diverse datasets and provides such a curated dataset for NER.

Figures (9)

• Figure 1: Country distribution. Distribution of country of origin for the articles in the full dataset (A) and in each of the splits (B-D). A and B share the color bar in the top row; C and D share the color bar in the bottom row. Detailed counts are available in Supplementary Tables \ref{['tab:countries-50']}-\ref{['tab:countries-125']}
• Figure 2: Dataset engineering. Procedure used for enforcing diversity and generating synthetic data for AskNews-NER-v0.
• Figure 2: Topics distribution. Percentage coverage of the 73 unique topics assigned to the collection of articles used for the AskNews-NER-v0 dataset. For legibility, the topics have been split up into three separate charts where A shows topics with $\geq$1% coverage and "other" corresponding to the remaining topics, B shows topics with $<$1% but $\geq$0.1% coverage and "other" corresponding to the remaining topics, and C shows topics with $<$0.1% coverage.
• Figure 3: Country representation. Country representation in the AskNews-NER-v0 dataset after enforcing country distribution, as per step 6 of the topic diversification pipeline.
• Figure 3: Entity type frequency for the raw labels. Labeling of news articles using Llama3 resulted in 182 unique entity types, despite requesting 54 as per Figure \ref{['box:types']}. Note that one of the entities types in the final 25% of the raw labels is an empty string.