Dialect2SQL: A Novel Text-to-SQL Dataset for Arabic Dialects with a Focus on Moroccan Darija

TL;DR

Dialect2SQL addresses the scarcity of high-quality text-to-SQL resources for Arabic dialects by introducing a large-scale, cross-domain Moroccan Darija dataset. It builds on the BIRD benchmark by translating its English questions into Darija via GPT-4 and subsequent manual refinement, yielding 9,428 NLQ-SQL pairs across 69 databases with complex schemas. The work also analyzes translation errors (CER, WER, TER) and provides baselines using code-generation LLMs (StarCoder2, CodeLlama, CodeT5), identifying StarCoder2-7B as the strongest performer on a Moroccan subset. This dataset supports future model development for low-resource languages and paves the way for expanding to additional dialects and bidirectional translation.

Abstract

The task of converting natural language questions (NLQs) into executable SQL queries, known as text-to-SQL, has gained significant interest in recent years, as it enables non-technical users to interact with relational databases. Many benchmarks, such as SPIDER and WikiSQL, have contributed to the development of new models and the evaluation of their performance. In addition, other datasets, like SEDE and BIRD, have introduced more challenges and complexities to better map real-world scenarios. However, these datasets primarily focus on high-resource languages such as English and Chinese. In this work, we introduce Dialect2SQL, the first large-scale, cross-domain text-to-SQL dataset in an Arabic dialect. It consists of 9,428 NLQ-SQL pairs across 69 databases in various domains. Along with SQL-related challenges such as long schemas, dirty values, and complex queries, our dataset also incorporates the complexities of the Moroccan dialect, which is known for its diverse source languages, numerous borrowed words, and unique expressions. This demonstrates that our dataset will be a valuable contribution to both the text-to-SQL community and the development of resources for low-resource languages.