Mamba-Diffusion Model with Learnable Wavelet for Controllable Symbolic Music Generation

TL;DR

This paper tackles the challenge of controllable symbolic music generation with diffusion models by representing music as image-like pianorolls and introducing a Mamba-Diffusion architecture that incorporates a Learnable Wavelet Node (LWN). The model combines a Transformer-Mamba block within a U-Net denoiser and uses cross-attention to inject chord information via a pre-trained chord VAE, trained with a diffusion objective and an auxiliary wavelet loss to preserve high-frequency details. Empirical results on POP909 show that Proffusion-WM achieves superior Overlapping Area (OA) and chord F1 scores compared with baselines, with strong human judgments on rhythm, harmony, and overall quality. This work advances controllable symbolic music generation by enabling precise chord-conditioned generation while maintaining high fidelity and interpretability of pianoroll representations, with code available for reproducibility.

Abstract

The recent surge in the popularity of diffusion models for image synthesis has attracted new attention to their potential for generation tasks in other domains. However, their applications to symbolic music generation remain largely under-explored because symbolic music is typically represented as sequences of discrete events and standard diffusion models are not well-suited for discrete data. We represent symbolic music as image-like pianorolls, facilitating the use of diffusion models for the generation of symbolic music. Moreover, this study introduces a novel diffusion model that incorporates our proposed Transformer-Mamba block and learnable wavelet transform. Classifier-free guidance is utilised to generate symbolic music with target chords. Our evaluation shows that our method achieves compelling results in terms of music quality and controllability, outperforming the strong baseline in pianoroll generation. Our code is available at https://github.com/jinchengzhanggg/proffusion.

Figures (5)

• Figure 1: Architecture of our proposed Mamba-Diffusion model with Wavelet for controllable music generation. Its denoising network is a U-Net combining learnable wavelet transform and our Transformer-Mamba blocks. Chords are encoded using a pre-trained VAE and fed into the U-Net via cross-attention layers.
• Figure 2: Illustration of the Learnable Wavelet Node (LWN).
• Figure 3: Pianoroll sample from the test set, with pitch on the vertical axis and time on the horizontal axis.
• Figure 4: Pianorolls generated by our proposed diffusion model and the other models.
• Figure 5: Boxplots of Overall Preference ratings for the four different models.