Rezwan: Leveraging Large Language Models for Comprehensive Hadith Text Processing: A 1.2M Corpus Development

TL;DR

Rezwan introduces a fully automated LLM-driven pipeline to construct a large-scale Hadith corpus (the Najm/Rezwan corpus) with over 1.5 million narrations and multilingual enrichment across 12 languages. Expert evaluation shows near-human accuracy for core tasks such as chain–text separation and abstractive summarization, with substantial advantages over the Noor Corpus due to rich enrichment layers. An economics-of-accuracy model quantifies the value of AI-assisted processing as equivalent to roughly $6.99\times 10^5$ expert-hours versus about $1.54\times 10^6$ manual-hours, highlighting dramatic cost and time savings. The work demonstrates a scalable, multilingual, semantically enriched resource for digital humanities and Islamic studies, enabling vast, research-ready access to Islamic heritage while inviting further domain-adapted refinements and tool development.

Abstract

This paper presents the development of Rezwan, a large-scale AI-assisted Hadith corpus comprising over 1.2M narrations, extracted and structured through a fully automated pipeline. Building on digital repositories such as Maktabat Ahl al-Bayt, the pipeline employs Large Language Models (LLMs) for segmentation, chain--text separation, validation, and multi-layer enrichment. Each narration is enhanced with machine translation into twelve languages, intelligent diacritization, abstractive summarization, thematic tagging, and cross-text semantic analysis. This multi-step process transforms raw text into a richly annotated research-ready infrastructure for digital humanities and Islamic studies. A rigorous evaluation was conducted on 1,213 randomly sampled narrations, assessed by six domain experts. Results show near-human accuracy in structured tasks such as chain--text separation (9.33/10) and summarization (9.33/10), while highlighting ongoing challenges in diacritization and semantic similarity detection. Comparative analysis against the manually curated Noor Corpus demonstrates the superiority of Najm in both scale and quality, with a mean overall score of 8.46/10 versus 3.66/10. Furthermore, cost analysis confirms the economic feasibility of the AI approach: tasks requiring over 229,000 hours of expert labor were completed within months at a fraction of the cost. The work introduces a new paradigm in religious text processing by showing how AI can augment human expertise, enabling large-scale, multilingual, and semantically enriched access to Islamic heritage.

Figures (3)

• Figure 1: Mean expert scores (0–10) for each evaluation aspect.
• Figure 2: Error rates (macro average) across major evaluation criteria.
• Figure 3: Strengths and limitations of the Rezwan Corpus across major aspects.