Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization

Usman Naseem, Juan Ren, Saba Anwar, Sarah Kohail, Rudy Alexandro Garrido Veliz, Robert Geislinger, Aisha Jabr, Idris Abdulmumin, Laiba Qureshi, Aarushi Ajay Borkar, Maryam Ibrahim Mukhtar, Abinew Ali Ayele, Ibrahim Said Ahmad, Adem Ali, Martin Semmann, Shamsuddeen Hassan Muhammad, Seid Muhie Yimam

TL;DR

POLAR tackles online polarization across languages, cultures, and events by introducing a large-scale, annotated benchmark with three detection tasks (binary presence, polarization type, and polarization manifestations). It collects data from seven languages and multiple platforms, and evaluates both multilingual pretrained models and large language models under monolingual, cross-lingual, and few-shot regimes. The study finds that binary polarization detection is reliably achievable with current models, whereas identifying nuanced types and manifestations remains difficult, especially in low-resource languages. By releasing POLAR and its benchmarks, the work aims to drive culturally robust NLP approaches and inform global strategies to mitigate digital polarization.

Abstract

Online polarization poses a growing challenge for democratic discourse, yet most computational social science research remains monolingual, culturally narrow, or event-specific. We introduce POLAR, a multilingual, multicultural, and multievent dataset with over 23k instances in seven languages from diverse online platforms and real-world events. Polarization is annotated along three axes: presence, type, and manifestation, using a variety of annotation platforms adapted to each cultural context. We conduct two main experiments: (1) we fine-tune six multilingual pretrained language models in both monolingual and cross-lingual setups; and (2) we evaluate a range of open and closed large language models (LLMs) in few-shot and zero-shot scenarios. Results show that while most models perform well on binary polarization detection, they achieve substantially lower scores when predicting polarization types and manifestations. These findings highlight the complex, highly contextual nature of polarization and the need for robust, adaptable approaches in NLP and computational social science. All resources will be released to support further research and effective mitigation of digital polarization globally.

POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization

TL;DR

POLAR tackles online polarization across languages, cultures, and events by introducing a large-scale, annotated benchmark with three detection tasks (binary presence, polarization type, and polarization manifestations). It collects data from seven languages and multiple platforms, and evaluates both multilingual pretrained models and large language models under monolingual, cross-lingual, and few-shot regimes. The study finds that binary polarization detection is reliably achievable with current models, whereas identifying nuanced types and manifestations remains difficult, especially in low-resource languages. By releasing POLAR and its benchmarks, the work aims to drive culturally robust NLP approaches and inform global strategies to mitigate digital polarization.

Abstract

Online polarization poses a growing challenge for democratic discourse, yet most computational social science research remains monolingual, culturally narrow, or event-specific. We introduce POLAR, a multilingual, multicultural, and multievent dataset with over 23k instances in seven languages from diverse online platforms and real-world events. Polarization is annotated along three axes: presence, type, and manifestation, using a variety of annotation platforms adapted to each cultural context. We conduct two main experiments: (1) we fine-tune six multilingual pretrained language models in both monolingual and cross-lingual setups; and (2) we evaluate a range of open and closed large language models (LLMs) in few-shot and zero-shot scenarios. Results show that while most models perform well on binary polarization detection, they achieve substantially lower scores when predicting polarization types and manifestations. These findings highlight the complex, highly contextual nature of polarization and the need for robust, adaptable approaches in NLP and computational social science. All resources will be released to support further research and effective mitigation of digital polarization globally.

Paper Structure

This paper contains 18 sections, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. POLAR Dataset Construction
  4. Operational Definitions
  5. Data Collection
  6. Annotation Process
  7. Dataset Characteristics
  8. Experimentation and Results
  9. Experimental Setup
  10. Results and Analysis
  11. Monolingual Setup Results
  12. Cross-Lingual MLM Results
  13. Zero and Few-shot LLM Performance
  14. Discussion
  15. Conclusion
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Pipeline for POLAR construction: multi-platform data curation in 7 languages, annotation workflow with quality control, and benchmarking.
  • Figure 2: Languages represented in the dataset, covering diverse linguistic and regional contexts.
  • Figure 3: Percentage of polarization per language