Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Detection of Somali-written Fake News and Toxic Messages on the Social Media Using Transformer-based Language Models

Muhidin A. Mohamed, Shuab D. Ahmed, Yahye A. Isse, Hanad M. Mohamed, Fuad M. Hassan, Houssein A. Assowe

TL;DR

This work tackles the scarcity of Somali NLP resources for misinformation and toxicity detection by building a large, diverse pre-training corpus (~160 million tokens) and creating labeled fake-news and toxicity datasets sourced from Facebook. It introduces SomBERTa, a monolingual Somali transformer model, and demonstrates its effectiveness through fine-tuning on fake-news, toxicity, and news-topic tasks, achieving an average accuracy of 87.99% and outperforming several multilingual baselines on key tasks. The study delivers a replicable methodology and publicly shareable resources designed to advance Somali NLP and support AI inclusivity for low-resource languages. Overall, the approach shows that language-specific models trained on targeted corpora can surpass multilingual counterparts in domain-specific Somali NLP applications, enabling scalable detection of harmful content for Somali-speaking communities.

Abstract

The fact that everyone with a social media account can create and share content, and the increasing public reliance on social media platforms as a news and information source bring about significant challenges such as misinformation, fake news, harmful content, etc. Although human content moderation may be useful to an extent and used by these platforms to flag posted materials, the use of AI models provides a more sustainable, scalable, and effective way to mitigate these harmful contents. However, low-resourced languages such as the Somali language face limitations in AI automation, including scarce annotated training datasets and lack of language models tailored to their unique linguistic characteristics. This paper presents part of our ongoing research work to bridge some of these gaps for the Somali language. In particular, we created two human-annotated social-media-sourced Somali datasets for two downstream applications, fake news \& toxicity classification, and developed a transformer-based monolingual Somali language model (named SomBERTa) -- the first of its kind to the best of our knowledge. SomBERTa is then fine-tuned and evaluated on toxic content, fake news and news topic classification datasets. Comparative evaluation analysis of the proposed model against related multilingual models (e.g., AfriBERTa, AfroXLMR, etc) demonstrated that SomBERTa consistently outperformed these comparators in both fake news and toxic content classification tasks while achieving the best average accuracy (87.99%) across all tasks. This research contributes to Somali NLP by offering a foundational language model and a replicable framework for other low-resource languages, promoting digital and AI inclusivity and linguistic diversity.

Detection of Somali-written Fake News and Toxic Messages on the Social Media Using Transformer-based Language Models

TL;DR

This work tackles the scarcity of Somali NLP resources for misinformation and toxicity detection by building a large, diverse pre-training corpus (~160 million tokens) and creating labeled fake-news and toxicity datasets sourced from Facebook. It introduces SomBERTa, a monolingual Somali transformer model, and demonstrates its effectiveness through fine-tuning on fake-news, toxicity, and news-topic tasks, achieving an average accuracy of 87.99% and outperforming several multilingual baselines on key tasks. The study delivers a replicable methodology and publicly shareable resources designed to advance Somali NLP and support AI inclusivity for low-resource languages. Overall, the approach shows that language-specific models trained on targeted corpora can surpass multilingual counterparts in domain-specific Somali NLP applications, enabling scalable detection of harmful content for Somali-speaking communities.

Abstract

The fact that everyone with a social media account can create and share content, and the increasing public reliance on social media platforms as a news and information source bring about significant challenges such as misinformation, fake news, harmful content, etc. Although human content moderation may be useful to an extent and used by these platforms to flag posted materials, the use of AI models provides a more sustainable, scalable, and effective way to mitigate these harmful contents. However, low-resourced languages such as the Somali language face limitations in AI automation, including scarce annotated training datasets and lack of language models tailored to their unique linguistic characteristics. This paper presents part of our ongoing research work to bridge some of these gaps for the Somali language. In particular, we created two human-annotated social-media-sourced Somali datasets for two downstream applications, fake news \& toxicity classification, and developed a transformer-based monolingual Somali language model (named SomBERTa) -- the first of its kind to the best of our knowledge. SomBERTa is then fine-tuned and evaluated on toxic content, fake news and news topic classification datasets. Comparative evaluation analysis of the proposed model against related multilingual models (e.g., AfriBERTa, AfroXLMR, etc) demonstrated that SomBERTa consistently outperformed these comparators in both fake news and toxic content classification tasks while achieving the best average accuracy (87.99%) across all tasks. This research contributes to Somali NLP by offering a foundational language model and a replicable framework for other low-resource languages, promoting digital and AI inclusivity and linguistic diversity.

Paper Structure

This paper contains 16 sections, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Pre-trained LLMs for low-resourced languages
  4. Multilingual LLMs covering the Somali language
  5. Research gaps and our proposed work
  6. Proposed transformer-based Somali language model
  7. Model Develepment Datasets
  8. Pre-training corpus
  9. Fake news and toxicity datasets
  10. Fake news dataset:
  11. Toxicity dataset:
  12. Model development
  13. Experimental evaluation
  14. Conclusion
  15. Ethical considerations
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Fake news and toxicity data collection and annotation. For each dataset, we used a set of pre-defined annotation criteria that all annotators applied (see Appendix A)
  • Figure 2: The distribution of the used fake and real news
  • Figure 3: Toxic(1) and non-toxic(0) proportions of the toxicity dataset
  • Figure 4: The unique words (vocabulary) for each of the merged datasets after preprocessing
  • Figure 5: Architecture and implementation methodology of the proposed transformer-based Somali fake news and toxicity classification model