Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ARCHI-TTS: A flow-matching-based Text-to-Speech Model with Self-supervised Semantic Aligner and Accelerated Inference

Chunyat Wu, Jiajun Deng, Zhengxi Liu, Zheqi Dai, Haolin He, Qiuqiang Kong

TL;DR

ARCHI-TTS tackles two main hurdles in zero-shot non-autoregressive TTS: aligning text with speech and the computational burden of iterative denoising. It introduces a semantic aligner to produce self-supervised, flexible-length text–speech representations and a flow-matching decoder that operates on compressed VAE latents, enabling fast, high-quality synthesis with CFG-guided sampling. The model further achieves significant inference speedups by reusing condition encoder outputs across denoising steps and adds an auxiliary CTC loss to reinforce alignment. Empirically, ARCHI-TTS delivers competitive WER and SSIM on LibriSpeech-PC and Seed-TTS with MOS rivaling industrial systems, while maintaining low compute requirements, highlighting its potential for multilingual, real-time TTS deployments.

Abstract

Although diffusion-based, non-autoregressive text-to-speech (TTS) systems have demonstrated impressive zero-shot synthesis capabilities, their efficacy is still hindered by two key challenges: the difficulty of text-speech alignment modeling and the high computational overhead of the iterative denoising process. To address these limitations, we propose ARCHI-TTS that features a dedicated semantic aligner to ensure robust temporal and semantic consistency between text and audio. To overcome high computational inference costs, ARCHI-TTS employs an efficient inference strategy that reuses encoder features across denoising steps, drastically accelerating synthesis without performance degradation. An auxiliary CTC loss applied to the condition encoder further enhances the semantic understanding. Experimental results demonstrate that ARCHI-TTS achieves a WER of 1.98% on LibriSpeech-PC test-clean, and 1.47%/1.42% on SeedTTS test-en/test-zh with a high inference efficiency, consistently outperforming recent state-of-the-art TTS systems.

ARCHI-TTS: A flow-matching-based Text-to-Speech Model with Self-supervised Semantic Aligner and Accelerated Inference

TL;DR

ARCHI-TTS tackles two main hurdles in zero-shot non-autoregressive TTS: aligning text with speech and the computational burden of iterative denoising. It introduces a semantic aligner to produce self-supervised, flexible-length text–speech representations and a flow-matching decoder that operates on compressed VAE latents, enabling fast, high-quality synthesis with CFG-guided sampling. The model further achieves significant inference speedups by reusing condition encoder outputs across denoising steps and adds an auxiliary CTC loss to reinforce alignment. Empirically, ARCHI-TTS delivers competitive WER and SSIM on LibriSpeech-PC and Seed-TTS with MOS rivaling industrial systems, while maintaining low compute requirements, highlighting its potential for multilingual, real-time TTS deployments.

Abstract

Although diffusion-based, non-autoregressive text-to-speech (TTS) systems have demonstrated impressive zero-shot synthesis capabilities, their efficacy is still hindered by two key challenges: the difficulty of text-speech alignment modeling and the high computational overhead of the iterative denoising process. To address these limitations, we propose ARCHI-TTS that features a dedicated semantic aligner to ensure robust temporal and semantic consistency between text and audio. To overcome high computational inference costs, ARCHI-TTS employs an efficient inference strategy that reuses encoder features across denoising steps, drastically accelerating synthesis without performance degradation. An auxiliary CTC loss applied to the condition encoder further enhances the semantic understanding. Experimental results demonstrate that ARCHI-TTS achieves a WER of 1.98% on LibriSpeech-PC test-clean, and 1.47%/1.42% on SeedTTS test-en/test-zh with a high inference efficiency, consistently outperforming recent state-of-the-art TTS systems.
Paper Structure (15 sections, 5 equations, 2 figures, 4 tables)

This paper contains 15 sections, 5 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. ARCHI-TTS
  3. Semantic Aligner
  4. Compressed Speech Latent Representation
  5. Condition Encoder with Flow Matching
  6. Velocity Decoder
  7. Zero-shot TTS
  8. Sampling
  9. Experimental Setup
  10. Datasets
  11. Model Configuration
  12. Baselines and Metrics
  13. Experimental Results
  14. Conclusion
  15. Acknowledgment

Figures (2)

  • Figure 1: The overview of the ARCHI-TTS architecture.
  • Figure 2: WER (solid line, left) and SSIM (dotted line, left), and RTF (right) across different sharing ratios.