Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DSFlow: Dual Supervision and Step-Aware Architecture for One-Step Flow Matching Speech Synthesis

Bin Lin, Peng Yang, Chao Yan, Xiaochen Liu, Wei Wang, Boyong Wu, Pengfei Tan, Xuerui Yang

TL;DR

DSFlow tackles the high latency of flow-matching speech synthesis by introducing a distillation framework that enables few-step and one-step generation. It combines dual supervision (endpoint and mean-velocity alignment), step-aware tokens for discrete inference, and weak classifier-free guidance to maintain controllability and stability without heavy Jacobian computations. Across multiple architectures and benchmarks, DSFlow achieves teacher-level quality in a single inference step while substantially reducing parameters and inference cost, and it preserves fine-grained prosody better than endpoint-only distillation. The work demonstrates that aligning architectural capacity with the reduced conditioning space of distilled tasks yields consistent gains and practical benefits for real-time text-to-speech systems.

Abstract

Flow-matching models have enabled high-quality text-to-speech synthesis, but their iterative sampling process during inference incurs substantial computational cost. Although distillation is widely used to reduce the number of inference steps, existing methods often suffer from process variance due to endpoint error accumulation. Moreover, directly reusing continuous-time architectures for discrete, fixed-step generation introduces structural parameter inefficiencies. To address these challenges, we introduce DSFlow, a modular distillation framework for few-step and one-step synthesis. DSFlow reformulates generation as a discrete prediction task and explicitly adapts the student model to the target inference regime. It improves training stability through a dual supervision strategy that combines endpoint matching with deterministic mean-velocity alignment, enforcing consistent generation trajectories across inference steps. In addition, DSFlow improves parameter efficiency by replacing continuous-time timestep conditioning with lightweight step-aware tokens, aligning model capacity with the significantly reduced timestep space of the discrete task. Extensive experiments across diverse flow-based text-to-speech architectures demonstrate that DSFlow consistently outperforms standard distillation approaches, achieving strong few-step and one-step synthesis quality while reducing model parameters and inference cost.

DSFlow: Dual Supervision and Step-Aware Architecture for One-Step Flow Matching Speech Synthesis

TL;DR

DSFlow tackles the high latency of flow-matching speech synthesis by introducing a distillation framework that enables few-step and one-step generation. It combines dual supervision (endpoint and mean-velocity alignment), step-aware tokens for discrete inference, and weak classifier-free guidance to maintain controllability and stability without heavy Jacobian computations. Across multiple architectures and benchmarks, DSFlow achieves teacher-level quality in a single inference step while substantially reducing parameters and inference cost, and it preserves fine-grained prosody better than endpoint-only distillation. The work demonstrates that aligning architectural capacity with the reduced conditioning space of distilled tasks yields consistent gains and practical benefits for real-time text-to-speech systems.

Abstract

Flow-matching models have enabled high-quality text-to-speech synthesis, but their iterative sampling process during inference incurs substantial computational cost. Although distillation is widely used to reduce the number of inference steps, existing methods often suffer from process variance due to endpoint error accumulation. Moreover, directly reusing continuous-time architectures for discrete, fixed-step generation introduces structural parameter inefficiencies. To address these challenges, we introduce DSFlow, a modular distillation framework for few-step and one-step synthesis. DSFlow reformulates generation as a discrete prediction task and explicitly adapts the student model to the target inference regime. It improves training stability through a dual supervision strategy that combines endpoint matching with deterministic mean-velocity alignment, enforcing consistent generation trajectories across inference steps. In addition, DSFlow improves parameter efficiency by replacing continuous-time timestep conditioning with lightweight step-aware tokens, aligning model capacity with the significantly reduced timestep space of the discrete task. Extensive experiments across diverse flow-based text-to-speech architectures demonstrate that DSFlow consistently outperforms standard distillation approaches, achieving strong few-step and one-step synthesis quality while reducing model parameters and inference cost.
Paper Structure (63 sections, 2 theorems, 20 equations, 3 figures, 8 tables, 1 algorithm)

This paper contains 63 sections, 2 theorems, 20 equations, 3 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Flow Matching and Efficient Inference.
  4. Distillation Methods for Flow-Based Models.
  5. Controllability in Distilled Generative Flow-Matching Models.
  6. Architectural Design for Distilled Models.
  7. Method
  8. Problem Formulation
  9. Endpoint Distillation.
  10. Dual Supervision for Process Alignment
  11. Mean Velocity Computation.
  12. Dual Supervision Loss.
  13. Step-Aware Tokenization for Structural Adaptation
  14. Information-Theoretic Motivation
  15. Redundancy of Continuous-Time Modulation
  16. ...and 48 more sections

Key Result

Proposition 3.1

For a model handling $K$ discrete inference steps with hidden dimension $D$ and $L$ Transformer layers, step-aware token conditioning introduces $O(KD)$ parameters, whereas adaLN-style conditioning introduces $O(LD^2)$ parameters.

Figures (3)

  • Figure 1: Overview of the DSFlow framework. The left and center panels show the transition from a DiT block with adaLN-Zero conditioning to the proposed step-aware architecture, where the heavy time-modulation network is replaced by lightweight step-aware tokens. The right panel illustrates dual supervision, which combines endpoint matching ($\mathcal{L}_{\mathrm{endpoint}}$) with deterministic mean velocity alignment ($\mathcal{L}_{\mathrm{velocity}}$) to guide the student along the teacher’s mean trajectory (green vectors), improving process consistency over endpoint-only distillation without additional Jacobian computation.
  • Figure 2: F0 distribution comparison. For both models, 100 speech samples were generated under the same text and prompt condition, and the frame-level F0 values of the generated samples are visualized via histograms and kernel density estimates.
  • Figure 3: Comparisons of key acoustic prosodic features between the teacher (10-step Flow Matching) and our student (1-step DSFlow) models. For each feature, 100 speech samples were generated under the same text and prompt condition, with frame-level values visualized via histograms and kernel density estimates. The student's distributions closely mirror the core mode of teacher while retaining natural variability—validating that our framework inherits the teacher's prosodic characteristics effectively.

Theorems & Definitions (3)

  • Proposition 3.1: Parameter Complexity of Step-Aware Conditioning
  • Proposition : Step-Aware Token Parameter Efficiency, restated
  • proof