D3RM: A Discrete Denoising Diffusion Refinement Model for Piano Transcription

TL;DR

This work introduces D3RM, a discrete denoising diffusion refinement model for piano transcription that conditions a Neighborhood Attention-based denoiser on a pretrained acoustic encoder. By representing note states as a $6$-way discrete vector (5 notes states plus [MASK]) and employing a novel absorbing-state sampling strategy, the model progressively refines the piano roll predictions with improved F1 scores on MAESTRO compared to baseline and DiffRoll, while remaining lightweight with around $2.0$M parameters. The approach leverages discrete diffusion theory, NA 2D attention, and cross-attention to fuse audio features, achieving competitive results with state-of-the-art methods and offering an efficient, refinement-driven alternative to purely feed-forward or continuous-diffusion methods. The findings highlight the practical potential of discrete diffusion refinements in music transcription and suggest future work on metric development and generalization to diverse acoustic models and token representations.

Abstract

Diffusion models have been widely used in the generative domain due to their convincing performance in modeling complex data distributions. Moreover, they have shown competitive results on discriminative tasks, such as image segmentation. While diffusion models have also been explored for automatic music transcription, their performance has yet to reach a competitive level. In this paper, we focus on discrete diffusion model's refinement capabilities and present a novel architecture for piano transcription. Our model utilizes Neighborhood Attention layers as the denoising module, gradually predicting the target high-resolution piano roll, conditioned on the finetuned features of a pretrained acoustic model. To further enhance refinement, we devise a novel strategy which applies distinct transition states during training and inference stage of discrete diffusion models. Experiments on the MAESTRO dataset show that our approach outperforms previous diffusion-based piano transcription models and the baseline model in terms of F1 score. Our code is available in https://github.com/hanshounsu/d3rm.

Figures (3)

• Figure 1: Overall model architecture. The yellow pixels within the piano rolls $\mathbf{y}_{\tau}$ represent mask states.
• Figure 2: Illustration of the overall discrete diffusion process including the absorbing state for the posterior $\tilde{q}$ in the inference stage. 5 states are visible in this example: off(0), offset(1), onset(2), sustain(3) and mask(M). Details for the posterior $\tilde{q}$ are described in \ref{['subsec:absorbing state']}.
• Figure 3: Transcription example from a single 10-second sample. In the case where $\overline{\gamma}_\mathcal{T}$ = 0.9, the model is trained with refinement and sampled using AS Sampling. In contrast, for $\overline{\gamma}_\mathcal{T}$ = 1.0, the training and inference stages use the absorbing state, meaning the model does not learn refinement. In the 3rd and 5th row figures, red-colored notes represent mispredicted notes, blue-colored notes indicate unpredicted notes, and grey-colored notes are those correctly predicted. The note errors are determined based on whether each predicted note's onset is correct.