Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control

TL;DR

Muse tackles the reproducibility bottleneck in long-form song generation by releasing a fully licensed synthetic dataset and an open-source generation model. It employs a simple, single-stage finetuning of a Qwen-based language model with MuCodec discrete audio tokens to model text and audio in a unified autoregressive sequence, enabling segment-level style conditioning. The dataset provides hierarchical global and segment-level style annotations, enabling fine-grained control and robust supervision. Results show competitive performance on lyric fidelity, style similarity, and audio aesthetics, while delivering a fully reproducible pipeline that facilitates fair comparisons and accelerated progress in controllable long-form music generation. Overall, Muse demonstrates that careful data design and minimal supervised optimization can achieve strong, reproducible baselines in a challenging, structure-rich domain.

Abstract

Recent commercial systems such as Suno demonstrate strong capabilities in long-form song generation, while academic research remains largely non-reproducible due to the lack of publicly available training data, hindering fair comparison and progress. To this end, we release a fully open-source system for long-form song generation with fine-grained style conditioning, including a licensed synthetic dataset, training and evaluation pipelines, and Muse, an easy-to-deploy song generation model. The dataset consists of 116k fully licensed synthetic songs with automatically generated lyrics and style descriptions paired with audio synthesized by SunoV5. We train Muse via single-stage supervised finetuning of a Qwen-based language model extended with discrete audio tokens using MuCodec, without task-specific losses, auxiliary objectives, or additional architectural components. Our evaluations find that although Muse is trained with a modest data scale and model size, it achieves competitive performance on phoneme error rate, text--music style similarity, and audio aesthetic quality, while enabling controllable segment-level generation across different musical structures. All data, model weights, and training and evaluation pipelines will be publicly released, paving the way for continued progress in controllable long-form song generation research. The project repository is available at https://github.com/yuhui1038/Muse.

Figures (4)

• Figure 1: Overview of Muse. The model operates on conversational, segment-structured inputs including global style labels, segment-level style descriptions, and lyrics. Text inputs are tokenized with a standard language tokenizer, and audio waveforms are encoded into discrete tokens via a neural audio codec. Text and audio tokens are unified into a single autoregressive sequence, enabling long-form song generation with segment-level style conditioning. [BOA]/[EOA] tokens mark audio boundaries, while [BOS]/[EOS] indicate full-sequence boundaries.
• Figure 2: Overview of the synthetic song data generation pipeline. GPT-5 mini generates structured prompts with global style labels and segmented lyrics, which are used by SunoV5 to synthesize full-length songs. Qwen3-Omni then produces hierarchical style annotations for each song.
• Figure 3: Duration distributions of Chinese and English songs in the dataset, showing comparable length profiles across languages.
• Figure 4: Comparison of overall audio quality across models. Left: Meta Audiobox Aesthetics scores. Right: SongEval scores. Higher values indicate better performance.