Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CIDAR: Culturally Relevant Instruction Dataset For Arabic

Zaid Alyafeai, Khalid Almubarak, Ahmed Ashraf, Deema Alnuhait, Saied Alshahrani, Gubran A. Q. Abdulrahman, Gamil Ahmed, Qais Gawah, Zead Saleh, Mustafa Ghaleb, Yousef Ali, Maged S. Al-Shaibani

TL;DR

CIDAR tackles Western-centric biases in Arabic instruction data by providing a culturally aligned, open 10k instruction-output dataset built from a translated seed and Arabic-specific prompts. It introduces a rigorous localization workflow with translation, manual review, and cultural adaptation, and evaluates fine-tuned models to demonstrate improved cultural alignment. The work highlights the importance of human review and cultural relevance for Arabic NLP resources, and shares code and data publicly to enable reproducibility and broader research. The dataset is poised to improve Arabic LLM performance across dialects, linguistics, and region-specific tasks.

Abstract

Instruction tuning has emerged as a prominent methodology for teaching Large Language Models (LLMs) to follow instructions. However, current instruction datasets predominantly cater to English or are derived from English-dominated LLMs, resulting in inherent biases toward Western culture. This bias significantly impacts the linguistic structures of non-English languages such as Arabic, which has a distinct grammar reflective of the diverse cultures across the Arab region. This paper addresses this limitation by introducing CIDAR: https://hf.co/datasets/arbml/CIDAR, the first open Arabic instruction-tuning dataset culturally-aligned by human reviewers. CIDAR contains 10,000 instruction and output pairs that represent the Arab region. We discuss the cultural relevance of CIDAR via the analysis and comparison to other models fine-tuned on other datasets. Our experiments show that CIDAR can help enrich research efforts in aligning LLMs with the Arabic culture. All the code is available at https://github.com/ARBML/CIDAR.

CIDAR: Culturally Relevant Instruction Dataset For Arabic

TL;DR

CIDAR tackles Western-centric biases in Arabic instruction data by providing a culturally aligned, open 10k instruction-output dataset built from a translated seed and Arabic-specific prompts. It introduces a rigorous localization workflow with translation, manual review, and cultural adaptation, and evaluates fine-tuned models to demonstrate improved cultural alignment. The work highlights the importance of human review and cultural relevance for Arabic NLP resources, and shares code and data publicly to enable reproducibility and broader research. The dataset is poised to improve Arabic LLM performance across dialects, linguistics, and region-specific tasks.

Abstract

Instruction tuning has emerged as a prominent methodology for teaching Large Language Models (LLMs) to follow instructions. However, current instruction datasets predominantly cater to English or are derived from English-dominated LLMs, resulting in inherent biases toward Western culture. This bias significantly impacts the linguistic structures of non-English languages such as Arabic, which has a distinct grammar reflective of the diverse cultures across the Arab region. This paper addresses this limitation by introducing CIDAR: https://hf.co/datasets/arbml/CIDAR, the first open Arabic instruction-tuning dataset culturally-aligned by human reviewers. CIDAR contains 10,000 instruction and output pairs that represent the Arab region. We discuss the cultural relevance of CIDAR via the analysis and comparison to other models fine-tuned on other datasets. Our experiments show that CIDAR can help enrich research efforts in aligning LLMs with the Arabic culture. All the code is available at https://github.com/ARBML/CIDAR.
Paper Structure (30 sections, 7 figures, 6 tables)

This paper contains 30 sections, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multilingual Instruction-tuning Datasets
  4. Arabic Instruction-tuning Datasets
  5. Issues of Arabic Instruction Datasets
  6. MTs-related Issues
  7. LLMs-related Issues
  8. Cidar
  9. Machine Translation
  10. Initial Review
  11. Localization
  12. Dataset Analysis
  13. Modifications
  14. Locations and Names
  15. Countries
  16. ...and 15 more sections

Figures (7)

  • Figure 1: An example of our localization procedure in Cidar of a given (HTML]FFF2CCinstruction, HTML]DEEBF7output) pair. We show, in colors, the grammatical and cultural modifications.
  • Figure 2: Workflow diagram of Cidar’s data collection pipeline, illustrating each pipeline phase and its components.
  • Figure 3: Number of mentions of every Arab country in both Cidar and translated AlpaGasus datasets.
  • Figure 4: Comparison between Cidar and translated AlpaGasus in terms of instruction (Left) and output (Right) lengths. Noticeably, the length of outputs increased in Cidar due to the possible reviewers' rewriting of outputs.
  • Figure 5: Comparison between Cidar and translated AlpaGasus datasets regarding the covered topics.
  • ...and 2 more figures