Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis

Yushen Chen, Junzhe Liu, Yujie Tu, Zhikang Niu, Yuzhe Liang, Kai Yu, Chunyu Qiang, Chen Zhang, Xie Chen

TL;DR

Habibi tackles the data scarcity and linguistic complexity of Arabic dialects by proposing an open-source unified-dialectal Arabic TTS framework that leverages diverse ASR data, a linguistically-informed curriculum, and dialect-aware fine-tuning. It introduces the first standardized multi-dialect Arabic zero-shot TTS benchmark, and demonstrates that a unified model can approach or exceed dialect-specific systems while outperforming a leading commercial baseline. The approach hinges on two-stage curriculum learning (MSA then dialectal data) and the use of regional identifiers during training and inference to improve dialect fidelity without diacritization. The work offers practical impact by providing open-source resources, a rigorous evaluation standard, and insights into data quality, coverage, and training strategies for long-tail multilingual TTS development.

Abstract

A notable gap persists in speech synthesis research and development for Arabic dialects, particularly from a unified modeling perspective. Despite its high practical value, the inherent linguistic complexity of Arabic dialects, further compounded by a lack of standardized data, benchmarks, and evaluation guidelines, steers researchers toward safer ground. To bridge this divide, we present Habibi, a suite of specialized and unified text-to-speech models that harnesses existing open-source ASR corpora to support a wide range of high- to low-resource Arabic dialects through linguistically-informed curriculum learning. Our approach outperforms the leading commercial service in generation quality, while maintaining extensibility through effective in-context learning, without requiring text diacritization. We are committed to open-sourcing the model, along with creating the first systematic benchmark for multi-dialect Arabic speech synthesis. Furthermore, by identifying the key challenges in and establishing evaluation standards for the process, we aim to provide a solid groundwork for subsequent research. Resources at https://SWivid.github.io/Habibi/ .

Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis

TL;DR

Habibi tackles the data scarcity and linguistic complexity of Arabic dialects by proposing an open-source unified-dialectal Arabic TTS framework that leverages diverse ASR data, a linguistically-informed curriculum, and dialect-aware fine-tuning. It introduces the first standardized multi-dialect Arabic zero-shot TTS benchmark, and demonstrates that a unified model can approach or exceed dialect-specific systems while outperforming a leading commercial baseline. The approach hinges on two-stage curriculum learning (MSA then dialectal data) and the use of regional identifiers during training and inference to improve dialect fidelity without diacritization. The work offers practical impact by providing open-source resources, a rigorous evaluation standard, and insights into data quality, coverage, and training strategies for long-tail multilingual TTS development.

Abstract

A notable gap persists in speech synthesis research and development for Arabic dialects, particularly from a unified modeling perspective. Despite its high practical value, the inherent linguistic complexity of Arabic dialects, further compounded by a lack of standardized data, benchmarks, and evaluation guidelines, steers researchers toward safer ground. To bridge this divide, we present Habibi, a suite of specialized and unified text-to-speech models that harnesses existing open-source ASR corpora to support a wide range of high- to low-resource Arabic dialects through linguistically-informed curriculum learning. Our approach outperforms the leading commercial service in generation quality, while maintaining extensibility through effective in-context learning, without requiring text diacritization. We are committed to open-sourcing the model, along with creating the first systematic benchmark for multi-dialect Arabic speech synthesis. Furthermore, by identifying the key challenges in and establishing evaluation standards for the process, we aim to provide a solid groundwork for subsequent research. Resources at https://SWivid.github.io/Habibi/ .
Paper Structure (23 sections, 1 equation, 1 figure, 13 tables)

This paper contains 23 sections, 1 equation, 1 figure, 13 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Terminology
  4. Training Data
  5. Multi-Dialect Arabic TTS Benchmark
  6. Linguistically-Informed Curriculum Learning
  7. Dialect-Aware Supervised Fine-Tuning
  8. Experiments
  9. Backbone Choice and Training Setup
  10. Evaluation Setup
  11. Results of Dialect-Specific Models
  12. Results of Unified-Dialectal Models
  13. Effectiveness of Curriculum Learning
  14. Effectiveness of In-Context Learning
  15. Ablation of Data Mixing and Scaling
  16. ...and 8 more sections

Figures (1)

  • Figure 1: Our open-source unified-dialectal model (Habibi) outperforms ElevenLabs' commercial TTS service: the latest Eleven v3 (alpha) (11Labs-3a).