Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Grounding Arabic LLMs in the Doha Historical Dictionary: Retrieval-Augmented Understanding of Quran and Hadith

Somaya Eltanbouly, Samer Rashwani

Abstract

Large language models (LLMs) have achieved remarkable progress in many language tasks, yet they continue to struggle with complex historical and religious Arabic texts such as the Quran and Hadith. To address this limitation, we develop a retrieval-augmented generation (RAG) framework grounded in diachronic lexicographic knowledge. Unlike prior RAG systems that rely on general-purpose corpora, our approach retrieves evidence from the Doha Historical Dictionary of Arabic (DHDA), a large-scale resource documenting the historical development of Arabic vocabulary. The proposed pipeline combines hybrid retrieval with an intent-based routing mechanism to provide LLMs with precise, contextually relevant historical information. Our experiments show that this approach improves the accuracy of Arabic-native LLMs, including Fanar and ALLaM, to over 85\%, substantially reducing the performance gap with Gemini, a proprietary large-scale model. Gemini also serves as an LLM-as-a-judge system for automatic evaluation in our experiments. The automated judgments were verified through human evaluation, demonstrating high agreement (kappa = 0.87). An error analysis further highlights key linguistic challenges, including diacritics and compound expressions. These findings demonstrate the value of integrating diachronic lexicographic resources into retrieval-augmented generation frameworks to enhance Arabic language understanding, particularly for historical and religious texts. The code and resources are publicly available at: https://github.com/somayaeltanbouly/Doha-Dictionary-RAG.

Grounding Arabic LLMs in the Doha Historical Dictionary: Retrieval-Augmented Understanding of Quran and Hadith

Abstract

Large language models (LLMs) have achieved remarkable progress in many language tasks, yet they continue to struggle with complex historical and religious Arabic texts such as the Quran and Hadith. To address this limitation, we develop a retrieval-augmented generation (RAG) framework grounded in diachronic lexicographic knowledge. Unlike prior RAG systems that rely on general-purpose corpora, our approach retrieves evidence from the Doha Historical Dictionary of Arabic (DHDA), a large-scale resource documenting the historical development of Arabic vocabulary. The proposed pipeline combines hybrid retrieval with an intent-based routing mechanism to provide LLMs with precise, contextually relevant historical information. Our experiments show that this approach improves the accuracy of Arabic-native LLMs, including Fanar and ALLaM, to over 85\%, substantially reducing the performance gap with Gemini, a proprietary large-scale model. Gemini also serves as an LLM-as-a-judge system for automatic evaluation in our experiments. The automated judgments were verified through human evaluation, demonstrating high agreement (kappa = 0.87). An error analysis further highlights key linguistic challenges, including diacritics and compound expressions. These findings demonstrate the value of integrating diachronic lexicographic resources into retrieval-augmented generation frameworks to enhance Arabic language understanding, particularly for historical and religious texts. The code and resources are publicly available at: https://github.com/somayaeltanbouly/Doha-Dictionary-RAG.
Paper Structure (26 sections, 3 figures, 7 tables)

This paper contains 26 sections, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Methodology
  4. Query Preprocessing
  5. Lexical Retrieval
  6. Neural Retrieval
  7. Re-ranking
  8. Intent-Based Structuring
  9. Generation
  10. Data Source and Dataset Construction
  11. Data Extraction
  12. Retrieval Corpus
  13. Data Generation
  14. Reranker Fine-Tuning Data
  15. Intent Classification Data
  16. ...and 11 more sections

Figures (3)

  • Figure 1: Overview of our RAG methodology. The pipeline consists of query analysis, retrieval, and re-ranking, and intent-driven routing for prompt construction. The LLM generates the final answer using a selected prompt template, where the template choice depends on the user intent extracted in the query analysis stage (e.g., $\mathbf{P_M}$: Meaning, $\mathbf{P_A}$: Author, $\mathbf{P_S}$: Source, etc.).
  • Figure 2: Distribution of absolute score differences between human evaluators and the Gemini 2.5 Pro automated judge ($N=200$). The chart demonstrates strong agreement, with 83% of evaluations resulting in an exact match (0-point difference) and over 95% deviated by no more than a single rubric category ($\leq$25 points).
  • Figure 3: Performance of Fanar and ALLaM across six question types, with the highest scores on Author of Citation and Contextual Meaning, and the lowest on Part of Speech.