Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ZAEBUC-Spoken: A Multilingual Multidialectal Arabic-English Speech Corpus

Injy Hamed, Fadhl Eryani, David Palfreyman, Nizar Habash

TL;DR

This work takes inspiration from established sets of transcription guidelines to present a set of guidelines handling issues of conversational speech, code-switching and orthography of both languages, and enrich the corpus with two layers of annotations.

Abstract

We present ZAEBUC-Spoken, a multilingual multidialectal Arabic-English speech corpus. The corpus comprises twelve hours of Zoom meetings involving multiple speakers role-playing a work situation where Students brainstorm ideas for a certain topic and then discuss it with an Interlocutor. The meetings cover different topics and are divided into phases with different language setups. The corpus presents a challenging set for automatic speech recognition (ASR), including two languages (Arabic and English) with Arabic spoken in multiple variants (Modern Standard Arabic, Gulf Arabic, and Egyptian Arabic) and English used with various accents. Adding to the complexity of the corpus, there is also code-switching between these languages and dialects. As part of our work, we take inspiration from established sets of transcription guidelines to present a set of guidelines handling issues of conversational speech, code-switching and orthography of both languages. We further enrich the corpus with two layers of annotations; (1) dialectness level annotation for the portion of the corpus where mixing occurs between different variants of Arabic, and (2) automatic morphological annotations, including tokenization, lemmatization, and part-of-speech tagging.

ZAEBUC-Spoken: A Multilingual Multidialectal Arabic-English Speech Corpus

TL;DR

This work takes inspiration from established sets of transcription guidelines to present a set of guidelines handling issues of conversational speech, code-switching and orthography of both languages, and enrich the corpus with two layers of annotations.

Abstract

We present ZAEBUC-Spoken, a multilingual multidialectal Arabic-English speech corpus. The corpus comprises twelve hours of Zoom meetings involving multiple speakers role-playing a work situation where Students brainstorm ideas for a certain topic and then discuss it with an Interlocutor. The meetings cover different topics and are divided into phases with different language setups. The corpus presents a challenging set for automatic speech recognition (ASR), including two languages (Arabic and English) with Arabic spoken in multiple variants (Modern Standard Arabic, Gulf Arabic, and Egyptian Arabic) and English used with various accents. Adding to the complexity of the corpus, there is also code-switching between these languages and dialects. As part of our work, we take inspiration from established sets of transcription guidelines to present a set of guidelines handling issues of conversational speech, code-switching and orthography of both languages. We further enrich the corpus with two layers of annotations; (1) dialectness level annotation for the portion of the corpus where mixing occurs between different variants of Arabic, and (2) automatic morphological annotations, including tokenization, lemmatization, and part-of-speech tagging.
Paper Structure (48 sections, 1 equation, 3 figures, 4 tables)

This paper contains 48 sections, 1 equation, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data Collection
  4. Overview of the Participants:
  5. Transcription
  6. Transcription Guidelines
  7. General Transcription Rules (GR)
  8. [GR-punctuation]
  9. [GR-numbers]$^H$
  10. [GR-background]$^H$
  11. [GR-unclear]$^H$
  12. [GR-mispronounciation]$^H$
  13. [GR-newTerm]$^H$
  14. Conversational Speech Rules (CR)
  15. [CR-repetition]$^H$
  16. ...and 33 more sections

Figures (3)

  • Figure 1: Example of transcription using Praat.
  • Figure 2: POS distribution for Arabic and English words in monolingual and code-switched (CSW) utterances.
  • Figure 3: Lemma cloud for the top-occurring 100 noun, verb, adjective, or adverb lemmas.