TidyVoice 2026 Challenge Evaluation Plan

TL;DR

The TidyVoice 2026 Challenge targets cross-lingual speaker verification under language mismatch by using the multilingual TidyVoiceX dataset derived from Mozilla Common Voice. It provides a reproducible benchmark with open training options, a clear evaluation protocol, and baselines to push toward language-robust speaker embeddings. Primary evaluation via Equal Error Rate and a secondary minDCF metric, across diverse language pairings and unseen languages, aims to promote fairer, language-independent recognition. Baseline results reveal language-dependent cues in current systems, underscoring the need for methods that generalize beyond language information and enabling practical improvements in multilingual speaker verification.

Abstract

The performance of speaker verification systems degrades significantly under language mismatch, a critical challenge exacerbated by the field's reliance on English-centric data. To address this, we propose the TidyVoice Challenge for cross-lingual speaker verification. The challenge leverages the TidyVoiceX dataset from the novel TidyVoice benchmark, a large-scale, multilingual corpus derived from Mozilla Common Voice, and specifically curated to isolate the effect of language switching across approximately 40 languages. Participants will be tasked with building systems robust to this mismatch, with performance primarily evaluated using the Equal Error Rate on cross-language trials. By providing standardized data, open-source baselines, and a rigorous evaluation protocol, this challenge aims to drive research towards fairer, more inclusive, and language-independent speaker recognition technologies, directly aligning with the Interspeech 2026 theme, "Speaking Together."

Figures (1)

• Figure 1: Score distributions for the four trial pair categories on the TidyVoiceX development set.