Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ArAIEval Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content

Maram Hasanain, Md. Arid Hasan, Fatema Ahmed, Reem Suwaileh, Md. Rafiul Biswas, Wajdi Zaghouani, Firoj Alam

TL;DR

This paper reports the second ArAIEval shared task focusing on propagandistic content in Arabic, across unimodal text and multimodal memes. Task 1 requires span-level detection of propagandistic techniques in tweets and news paragraphs, while Task 2 classifies memes across text, image, and multimodal modalities. The datasets are manually annotated with a 23-technique taxonomy and are released alongside evaluation scripts, enabling robust benchmarking. Across tasks, transformer-based Arabic pretrained models dominated submissions, delivering performance well above baselines and underscoring the practical impact of Arabic-focused propaganda detection research.

Abstract

We present an overview of the second edition of the ArAIEval shared task, organized as part of the ArabicNLP 2024 conference co-located with ACL 2024. In this edition, ArAIEval offers two tasks: (i) detection of propagandistic textual spans with persuasion techniques identification in tweets and news articles, and (ii) distinguishing between propagandistic and non-propagandistic memes. A total of 14 teams participated in the final evaluation phase, with 6 and 9 teams participating in Tasks 1 and 2, respectively. Finally, 11 teams submitted system description papers. Across both tasks, we observed that fine-tuning transformer models such as AraBERT was at the core of the majority of the participating systems. We provide a description of the task setup, including a description of the dataset construction and the evaluation setup. We further provide a brief overview of the participating systems. All datasets and evaluation scripts are released to the research community (https://araieval.gitlab.io/). We hope this will enable further research on these important tasks in Arabic.

ArAIEval Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content

TL;DR

This paper reports the second ArAIEval shared task focusing on propagandistic content in Arabic, across unimodal text and multimodal memes. Task 1 requires span-level detection of propagandistic techniques in tweets and news paragraphs, while Task 2 classifies memes across text, image, and multimodal modalities. The datasets are manually annotated with a 23-technique taxonomy and are released alongside evaluation scripts, enabling robust benchmarking. Across tasks, transformer-based Arabic pretrained models dominated submissions, delivering performance well above baselines and underscoring the practical impact of Arabic-focused propaganda detection research.

Abstract

We present an overview of the second edition of the ArAIEval shared task, organized as part of the ArabicNLP 2024 conference co-located with ACL 2024. In this edition, ArAIEval offers two tasks: (i) detection of propagandistic textual spans with persuasion techniques identification in tweets and news articles, and (ii) distinguishing between propagandistic and non-propagandistic memes. A total of 14 teams participated in the final evaluation phase, with 6 and 9 teams participating in Tasks 1 and 2, respectively. Finally, 11 teams submitted system description papers. Across both tasks, we observed that fine-tuning transformer models such as AraBERT was at the core of the majority of the participating systems. We provide a description of the task setup, including a description of the dataset construction and the evaluation setup. We further provide a brief overview of the participating systems. All datasets and evaluation scripts are released to the research community (https://araieval.gitlab.io/). We hope this will enable further research on these important tasks in Arabic.
Paper Structure (21 sections, 2 figures, 6 tables)

This paper contains 21 sections, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Persuasion Techniques
  4. Multimodal Detection
  5. Related Shared Tasks
  6. Task 1: Unimodal (Text) Propagandistic Technique Detection
  7. Task Definition
  8. Data
  9. Data annotation:
  10. Data splits:
  11. Statistics:
  12. Evaluation Setup
  13. Results and Overview of the Systems
  14. Task 2: Multimodal Propagandistic Memes Classification
  15. Task Definition
  16. ...and 6 more sections

Figures (2)

  • Figure 1: An example of a news paragraph annotated with propagandistic techniques.
  • Figure 2: An example of memes with propagandistic and not-propagandistic categories.