Step-Audio-EditX Technical Report
Chao Yan, Boyong Wu, Peng Yang, Pengfei Tan, Guoqiang Hu, Li Xie, Yuxin Zhang, Xiangyu, Zhang, Fei Tian, Xuerui Yang, Xiangyu Zhang, Daxin Jiang, Shuchang Zhou, Gang Yu
TL;DR
Step-Audio-EditX introduces an open-source LLM-based audio model capable of expressive, iterative editing of emotion, speaking style, and paralinguistics while delivering robust zero-shot TTS. It hinges on large-margin synthetic data to achieve attribute disentanglement and iterative control without extra adapters, leveraging a dual-codebook audio tokenizer, a 3B audio LLM initialized from a text LLM, and a flow-based decoder. The system attains superior emotion and style editing and effective paralinguistic editing, with strong generalization to closed-source TTS systems and extensible capabilities like speed editing and denoising. By prioritizing data-driven attribute control over representation disentanglement, the approach offers practical impact for expressive speech synthesis and editing workflows and suggests avenues for broader vocal editing tasks.
Abstract
We present Step-Audio-EditX, the first open-source LLM-based audio model excelling at expressive and iterative audio editing encompassing emotion, speaking style, and paralinguistics alongside robust zero-shot text-to-speech (TTS) capabilities. Our core innovation lies in leveraging only large-margin synthetic data, which circumvents the need for embedding-based priors or auxiliary modules. This large-margin learning approach enables both iterative control and high expressivity across voices, and represents a fundamental pivot from the conventional focus on representation-level disentanglement. Evaluation results demonstrate that Step-Audio-EditX surpasses both MiniMax-2.6-hd and Doubao-Seed-TTS-2.0 in emotion editing and other fine-grained control tasks.