MeanAudio: Fast and Faithful Text-to-Audio Generation with Mean Flows

TL;DR

MeanAudio introduces a MeanFlow-based framework for fast, faithful text-to-audio generation, achieving 1-NFE with an ultra-low real-time factor of 0.013 on a single RTX 3090 and a 100x speedup over diffusion baselines. It leverages an enhanced Flux-Style transformer with dual text encoders, audio latent encoding via a VAE, and integrated classifier-free guidance trained into the objective, enabling efficient single-step and competitive multi-step generation. A two-stage instantaneous-to-mean curriculum with flow-field mix-up stabilizes training and accelerates convergence, with extensive ablations highlighting the importance of architectural choices and training strategies. The approach scales with data and model size and demonstrates strong performance on AudioCaps and MusicCaps, offering practical implications for real-time, high-quality TTA systems while outlining limitations related to fixed-length segments and speech-level capabilities.

Abstract

Recent years have witnessed remarkable progress in Text-to-Audio Generation (TTA), providing sound creators with powerful tools to transform inspirations into vivid audio. Yet despite these advances, current TTA systems often suffer from slow inference speed, which greatly hinders the efficiency and smoothness of audio creation. In this paper, we present MeanAudio, a fast and faithful text-to-audio generator capable of rendering realistic sound with only one function evaluation (1-NFE). MeanAudio leverages: (i) the MeanFlow objective with guided velocity target that significantly accelerates inference speed, (ii) an enhanced Flux-style transformer with dual text encoders for better semantic alignment and synthesis quality, and (iii) an efficient instantaneous-to-mean curriculum that speeds up convergence and enables training on consumer-grade GPUs. Through a comprehensive evaluation study, we demonstrate that MeanAudio achieves state-of-the-art performance in single-step audio generation. Specifically, it achieves a real-time factor (RTF) of 0.013 on a single NVIDIA RTX 3090, yielding a 100x speedup over SOTA diffusion-based TTA systems. Moreover, MeanAudio also shows strong performance in multi-step generation, enabling smooth transitions across successive synthesis steps.

Figures (4)

• Figure 1: MeanAudio achieves state-of-the-art single-step generation performance with a real-time factor (RTF) of 0.013, offering a 100x speedup over existing diffusion-based TTA systems. It also demonstrates strong performance in multi-step generation, despite using only 120M parameters.
• Figure 2: Model architecture overview: MeanAudio combines $N_1$ multi-modal (MMDiT) blocks and $N_2$ single-modal (DiT) blocks to construct the flow transformer. It leverages the joint attention in multi-modal blocks to integrate FLAN-T5’s fine-grained text embeddings, and employs AdaLN to inject CLAP’s global conditioning.
• Figure 3: Illustration of the underlying instantaneous field $v_t$, which models tangents of the flow trajectory, and the average velocity field $u(x_t, r, t)$, which captures long displacements. Our training curriculum encourages the model to first learn the foundational instantaneous dynamics, then gradually adapt to mean flows for fast and faithful generation.
• Figure 4: Screenshot of the subjective evaluation platform.