WenetSpeech-Wu: Datasets, Benchmarks, and Models for a Unified Chinese Wu Dialect Speech Processing Ecosystem

TL;DR

This work tackles the under-resourced Wu Chinese dialect in speech processing by introducing WenetSpeech-Wu, the first large-scale open-source Wu corpus (~8,000 hours) with multi-dimensional annotations, and WenetSpeech-Wu-Bench, a public benchmark for ASR, Wu-to-Mandarin translation, speaker attributes, emotion recognition, TTS, and instruct TTS. It details a scalable data construction pipeline, including automatic transcription fusion and rich paralinguistic annotations, to enable broad downstream tasks. The authors also release strong open-source models for ASR and TTS, plus unified understanding and instruct TTS models trained on the Wu corpus, achieving state-of-the-art or competitive results across tasks on the Wu benchmark. Together, the dataset, benchmark, and models establish a foundational Wu dialect speech processing ecosystem with potential for accelerating dialectal AI research and real-world deployment.

Abstract

Speech processing for low-resource dialects remains a fundamental challenge in developing inclusive and robust speech technologies. Despite its linguistic significance and large speaker population, the Wu dialect of Chinese has long been hindered by the lack of large-scale speech data, standardized evaluation benchmarks, and publicly available models. In this work, we present WenetSpeech-Wu, the first large-scale, multi-dimensionally annotated open-source speech corpus for the Wu dialect, comprising approximately 8,000 hours of diverse speech data. Building upon this dataset, we introduce WenetSpeech-Wu-Bench, the first standardized and publicly accessible benchmark for systematic evaluation of Wu dialect speech processing, covering automatic speech recognition (ASR), Wu-to-Mandarin translation, speaker attribute prediction, speech emotion recognition, text-to-speech (TTS) synthesis, and instruction-following TTS (instruct TTS). Furthermore, we release a suite of strong open-source models trained on WenetSpeech-Wu, establishing competitive performance across multiple tasks and empirically validating the effectiveness of the proposed dataset. Together, these contributions lay the foundation for a comprehensive Wu dialect speech processing ecosystem, and we open-source proposed datasets, benchmarks, and models to support future research on dialectal speech intelligence.

Figures (5)

• Figure 1: Highlights of WenetSpeech-Wu.
• Figure 2: Data construction pipeline for WenetSpeech-Wu.
• Figure 3: Statistical overview of WenetSpeech-Wu.
• Figure 4: Example of an Annotated Data Entry in JSON format
• Figure 5: Supplementary comparison results on age, gender, and emotion recognition tasks.