Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Matcha-TTS: A fast TTS architecture with conditional flow matching

Shivam Mehta, Ruibo Tu, Jonas Beskow, Éva Székely, Gustav Eje Henter

TL;DR

Matcha-TTS tackles slow diffusion-based TTS by introducing a fast, non-autoregressive encoder-decoder trained with optimal-transport conditional flow matching (OT-CFM). The architecture combines a RoPE-enabled encoder with a 1D U-Net decoder and a straightforward, linear gradient flow, enabling high-quality speech with far fewer steps and reduced memory. Empirical results show superior naturalness (MOS) and competitive speed, especially on longer utterances, compared to strong baselines. The work demonstrates that OT-CFM together with a memory-efficient decoder yields practical, high-quality TTS with reduced computation and latency, without external alignments.

Abstract

We introduce Matcha-TTS, a new encoder-decoder architecture for speedy TTS acoustic modelling, trained using optimal-transport conditional flow matching (OT-CFM). This yields an ODE-based decoder capable of high output quality in fewer synthesis steps than models trained using score matching. Careful design choices additionally ensure each synthesis step is fast to run. The method is probabilistic, non-autoregressive, and learns to speak from scratch without external alignments. Compared to strong pre-trained baseline models, the Matcha-TTS system has the smallest memory footprint, rivals the speed of the fastest models on long utterances, and attains the highest mean opinion score in a listening test. Please see https://shivammehta25.github.io/Matcha-TTS/ for audio examples, code, and pre-trained models.

Matcha-TTS: A fast TTS architecture with conditional flow matching

TL;DR

Matcha-TTS tackles slow diffusion-based TTS by introducing a fast, non-autoregressive encoder-decoder trained with optimal-transport conditional flow matching (OT-CFM). The architecture combines a RoPE-enabled encoder with a 1D U-Net decoder and a straightforward, linear gradient flow, enabling high-quality speech with far fewer steps and reduced memory. Empirical results show superior naturalness (MOS) and competitive speed, especially on longer utterances, compared to strong baselines. The work demonstrates that OT-CFM together with a memory-efficient decoder yields practical, high-quality TTS with reduced computation and latency, without external alignments.

Abstract

We introduce Matcha-TTS, a new encoder-decoder architecture for speedy TTS acoustic modelling, trained using optimal-transport conditional flow matching (OT-CFM). This yields an ODE-based decoder capable of high output quality in fewer synthesis steps than models trained using score matching. Careful design choices additionally ensure each synthesis step is fast to run. The method is probabilistic, non-autoregressive, and learns to speak from scratch without external alignments. Compared to strong pre-trained baseline models, the Matcha-TTS system has the smallest memory footprint, rivals the speed of the fastest models on long utterances, and attains the highest mean opinion score in a listening test. Please see https://shivammehta25.github.io/Matcha-TTS/ for audio examples, code, and pre-trained models.
Paper Structure (11 sections, 4 equations, 3 figures, 1 table)

This paper contains 11 sections, 4 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Recent encoder-decoder TTS architectures
  4. Flow matching and TTS
  5. Method
  6. Optimal-transport conditional flow matching
  7. Proposed architecture
  8. Experiments
  9. Data and systems
  10. Evaluations, results, and discussion
  11. Conclusions and future work

Figures (3)

  • Figure 1: Overview of the proposed approach at synthesis time.
  • Figure 2: Matcha-TTS decoder (the flow-prediction network in \ref{['fig: model architecture overview']}).
  • Figure 3: Scatterplot of prompt length vs. synthesis time for acoustic models. Regression lines show as curves due to the log-log axes.