Sidon: Fast and Robust Open-Source Multilingual Speech Restoration for Large-scale Dataset Cleansing

TL;DR

Sidon addresses the need for clean multilingual speech data to improve TTS by introducing an open-source two-component restoration pipeline based on a w2v-BERT 2.0 feature predictor and a HiFi-GAN vocoder. Trained on 2,219 hours across 104 languages with a three-stage process, Sidon achieves restoration quality comparable to the closed-source Miipher while offering broad language coverage and rapid inference. Empirical results show Sidon enhances downstream TTS when used to cleanse in-the-wild ASR data and delivers substantial speed gains, enabling scalable dataset cleansing of large corpora. This work lowers barriers to high-quality multilingual TTS research by providing reproducible, open-source tooling and demonstrating the practical impact of dataset cleansing on synthetic speech quality.

Abstract

Large-scale text-to-speech (TTS) systems are limited by the scarcity of clean, multilingual recordings. We introduce Sidon, a fast, open-source speech restoration model that converts noisy in-the-wild speech into studio-quality speech and scales to dozens of languages. Sidon consists of two models: w2v-BERT 2.0 finetuned feature predictor to cleanse features from noisy speech and vocoder trained to synthesize restored speech from the cleansed features. Sidon achieves restoration performance comparable to Miipher: Google's internal speech restoration model with the aim of dataset cleansing for speech synthesis. Sidon is also computationally efficient, running up to 500 times faster than real time on a single GPU. We further show that training a TTS model using a Sidon-cleansed automatic speech recognition corpus improves the quality of synthetic speech in a zero-shot setting. Code and model are released to facilitate reproducible dataset cleansing for the research community.

Figures (1)

• Figure 1: Overall architecture of Sidon