MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encoder

TL;DR

MiniMax-Speech tackles robust zero-shot voice cloning and high-quality multilingual TTS by marrying a learnable speaker encoder with an autoregressive Transformer and a Flow-VAE-based latent flow matching. The learnable encoder enables text-free, cross-lingual voice cloning from untranscribed reference audio, while Flow-VAE improves latent representations and speaker similarity. Across 32 languages and multiple objective/subjective metrics, it achieves SOTA voice-cloning scores and tops the public TTS Arena leaderboard; extensions include emotion control, T2V, and professional voice cloning. The approach offers strong cross-lingual generalization and a flexible foundation for downstream, parameter-efficient customization.

Abstract

We introduce MiniMax-Speech, an autoregressive Transformer-based Text-to-Speech (TTS) model that generates high-quality speech. A key innovation is our learnable speaker encoder, which extracts timbre features from a reference audio without requiring its transcription. This enables MiniMax-Speech to produce highly expressive speech with timbre consistent with the reference in a zero-shot manner, while also supporting one-shot voice cloning with exceptionally high similarity to the reference voice. In addition, the overall quality of the synthesized audio is enhanced through the proposed Flow-VAE. Our model supports 32 languages and demonstrates excellent performance across multiple objective and subjective evaluations metrics. Notably, it achieves state-of-the-art (SOTA) results on objective voice cloning metrics (Word Error Rate and Speaker Similarity) and has secured the top position on the public TTS Arena leaderboard. Another key strength of MiniMax-Speech, granted by the robust and disentangled representations from the speaker encoder, is its extensibility without modifying the base model, enabling various applications such as: arbitrary voice emotion control via LoRA; text to voice (T2V) by synthesizing timbre features directly from text description; and professional voice cloning (PVC) by fine-tuning timbre features with additional data. We encourage readers to visit https://minimax-ai.github.io/tts_tech_report for more examples.

Figures (5)

• Figure 1: An overview of the architecture of MiniMax-Speech.
• Figure 2: Different Voice Cloning Approaches in AR Transformer. The dotted line represents a provided example of a text-to-speech pair.
• Figure 3: An overview of the proposed latent flow matching Architecture. (a) The Flow-VAE model consists of an encoder, which is used to extract continuous speech features $z$, and a decoder, which is used to restore continuous speech features back to waveforms, and a flow model, which converts the distribution of continuous speech features to a standard normal distribution. $z$ represents continuous speech features, which is the target for the flow matching model to generate. (b) The flow matching model, which conditions on AR transformer output $c$, a speaker embedding $v$, provided continuous speech features $x_p$ and intermediate state $x_t$ at timestep $t$ on the probabilistic density path. $c'$ represents the output of AR transformer after upsampling. The dotted box indicates the provided prompt information.
• Figure 4: Artificial Arena Evaluation Results (May 12, 2025).
• Figure :