Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Constant in HATE: Analyzing Toxicity in Reddit across Topics and Languages

Wondimagegnhue Tsegaye Tufa, Ilia Markov, Piek Vossen

TL;DR

This paper tackles toxicity in online discourse by analyzing Reddit through a cross-topic and cross-lingual lens. It builds a large multilingual dataset of 1.5 million comment threads across 481 communities in six languages (English, German, Spanish, Turkish, Arabic, Dutch) and applies three toxicity scoring approaches (lexicon-based, GPT-4, Perspective API) plus expert annotations. The results show politics and world news topics are consistently more toxic across languages, while language-specific patterns reveal both cross-language similarities and meaningful differences in toxicity profiles. The work provides a resource and methodological insights to improve moderation and multilingual toxicity detection in NLP models.

Abstract

Toxic language remains an ongoing challenge on social media platforms, presenting significant issues for users and communities. This paper provides a cross-topic and cross-lingual analysis of toxicity in Reddit conversations. We collect 1.5 million comment threads from 481 communities in six languages: English, German, Spanish, Turkish,Arabic, and Dutch, covering 80 topics such as Culture, Politics, and News. We thoroughly analyze how toxicity spikes within different communities in relation to specific topics. We observe consistent patterns of increased toxicity across languages for certain topics, while also noting significant variations within specific language communities.

The Constant in HATE: Analyzing Toxicity in Reddit across Topics and Languages

TL;DR

This paper tackles toxicity in online discourse by analyzing Reddit through a cross-topic and cross-lingual lens. It builds a large multilingual dataset of 1.5 million comment threads across 481 communities in six languages (English, German, Spanish, Turkish, Arabic, Dutch) and applies three toxicity scoring approaches (lexicon-based, GPT-4, Perspective API) plus expert annotations. The results show politics and world news topics are consistently more toxic across languages, while language-specific patterns reveal both cross-language similarities and meaningful differences in toxicity profiles. The work provides a resource and methodological insights to improve moderation and multilingual toxicity detection in NLP models.

Abstract

Toxic language remains an ongoing challenge on social media platforms, presenting significant issues for users and communities. This paper provides a cross-topic and cross-lingual analysis of toxicity in Reddit conversations. We collect 1.5 million comment threads from 481 communities in six languages: English, German, Spanish, Turkish,Arabic, and Dutch, covering 80 topics such as Culture, Politics, and News. We thoroughly analyze how toxicity spikes within different communities in relation to specific topics. We observe consistent patterns of increased toxicity across languages for certain topics, while also noting significant variations within specific language communities.
Paper Structure (34 sections, 4 figures, 3 tables)

This paper contains 34 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Comment and post analysis
  4. Community analysis
  5. Toxic language resource
  6. Measuring toxicity
  7. Topic and language analysis
  8. Methodology
  9. Data source
  10. Data collection and preprocessing
  11. Language detection
  12. Topic identification
  13. Pre-processing
  14. Toxicity scores
  15. Lexicon-based approach
  16. ...and 19 more sections

Figures (4)

  • Figure 1: Comparison of toxicity levels in Reddit discussions across different topics and languages. The scores represent the toxicity density, the proportion of toxic comments within each topic. Each line illustrates the toxicity density for a specific language within a particular topic.
  • Figure 2: Distribution of toxic comments across topics based on the lexical-based approach. For visibility, we show the top 15 topics. Here, a comment is considered toxic if it contains at least one toxic word.
  • Figure 3: Distribution of thread toxicity across topics. For visibility, we only show the top 10 topics. Plots are sorted by the mean value.
  • Figure 4: Toxicity scores using the lexicon-based approach. The number under each language shows the total number of lexicon entries in that language.