CS-Dialogue: A 104-Hour Dataset of Spontaneous Mandarin-English Code-Switching Dialogues for Speech Recognition

TL;DR

CS-Dialogue introduces a large-scale, publicly available Mandarin-English code-switching dataset comprising 104.02 hours of spontaneous dialogues from 200 speakers, with full transcripts and cross-language coverage. The authors detail meticulous data collection, annotation, and quality-control pipelines, and establish benchmark ASR performance using both scratch-trained models (Transformer, Conformer, Branchformer) and multiple pre-trained models (Whisper variants, Qwen2-Audio, SenseVoice-Small, Paraformer). Results reveal substantial code-switching challenges for ASR, with fine-tuning yielding notable gains and topic-dependent variability in error rates. The dataset provides a robust, open resource to advance code-switching ASR research and dialogue modeling in multilingual contexts.

Abstract

Code-switching (CS), the alternation between two or more languages within a single conversation, presents significant challenges for automatic speech recognition (ASR) systems. Existing Mandarin-English code-switching datasets often suffer from limitations in size, spontaneity, and the lack of full-length dialogue recordings with transcriptions, hindering the development of robust ASR models for real-world conversational scenarios. This paper introduces CS-Dialogue, a novel large-scale Mandarin-English code-switching speech dataset comprising 104 hours of spontaneous conversations from 200 speakers. Unlike previous datasets, CS-Dialogue provides full-length dialogue recordings with complete transcriptions, capturing naturalistic code-switching patterns in continuous speech. We describe the data collection and annotation processes, present detailed statistics of the dataset, and establish benchmark ASR performance using state-of-the-art models. Our experiments, using Transformer, Conformer, and Branchformer, demonstrate the challenges of code-switching ASR, and show that existing pre-trained models such as Whisper still have the space to improve. The CS-Dialogue dataset will be made freely available for all academic purposes.

Figures (8)

• Figure 1: Distribution of speakers by age and gender
• Figure 2: Distribution of speakers by region
• Figure 3: Utterance-level (left) and speaker-level (right) duration distributions.
• Figure 4: Duration of each language per data split
• Figure 5: Comparison of MER for Whisper Large-V2, Qwen2-Audio, Paraformer, and SenseVoice-Small Across Different Conversation Topics