Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Realistic Synthetic Data for Automatic Drum Transcription

Pierfrancesco Melucci, Paolo Merialdo, Taketo Akama

TL;DR

This work tackles the data scarcity and domain gap in automatic drum transcription by introducing a semi-supervised pipeline that automatically curates a large one-shot drum-sample library from unlabeled audio using CLAP embeddings. The library enables on-the-fly synthesis of a high-quality training corpus from MIDI, which is used to train a sequence-to-sequence Transformer for audio-to-MIDI transcription. The approach achieves state-of-the-art results on ENST and MDB, outperforming both fully supervised and prior synthetic-data methods, and demonstrates the value of a 26-class instrument vocabulary derived from the MIDI percussion map. Overall, the data-centric method reduces dependence on paired datasets and improves generalization to real drum recordings, with code and resources publicly available.

Abstract

Deep learning models define the state-of-the-art in Automatic Drum Transcription (ADT), yet their performance is contingent upon large-scale, paired audio-MIDI datasets, which are scarce. Existing workarounds that use synthetic data often introduce a significant domain gap, as they typically rely on low-fidelity SoundFont libraries that lack acoustic diversity. While high-quality one-shot samples offer a better alternative, they are not available in a standardized, large-scale format suitable for training. This paper introduces a new paradigm for ADT that circumvents the need for paired audio-MIDI training data. Our primary contribution is a semi-supervised method to automatically curate a large and diverse corpus of one-shot drum samples from unlabeled audio sources. We then use this corpus to synthesize a high-quality dataset from MIDI files alone, which we use to train a sequence-to-sequence transcription model. We evaluate our model on the ENST and MDB test sets, where it achieves new state-of-the-art results, significantly outperforming both fully supervised methods and previous synthetic-data approaches. The code for reproducing our experiments is publicly available at https://github.com/pier-maker92/ADT_STR

Towards Realistic Synthetic Data for Automatic Drum Transcription

TL;DR

This work tackles the data scarcity and domain gap in automatic drum transcription by introducing a semi-supervised pipeline that automatically curates a large one-shot drum-sample library from unlabeled audio using CLAP embeddings. The library enables on-the-fly synthesis of a high-quality training corpus from MIDI, which is used to train a sequence-to-sequence Transformer for audio-to-MIDI transcription. The approach achieves state-of-the-art results on ENST and MDB, outperforming both fully supervised and prior synthetic-data methods, and demonstrates the value of a 26-class instrument vocabulary derived from the MIDI percussion map. Overall, the data-centric method reduces dependence on paired datasets and improves generalization to real drum recordings, with code and resources publicly available.

Abstract

Deep learning models define the state-of-the-art in Automatic Drum Transcription (ADT), yet their performance is contingent upon large-scale, paired audio-MIDI datasets, which are scarce. Existing workarounds that use synthetic data often introduce a significant domain gap, as they typically rely on low-fidelity SoundFont libraries that lack acoustic diversity. While high-quality one-shot samples offer a better alternative, they are not available in a standardized, large-scale format suitable for training. This paper introduces a new paradigm for ADT that circumvents the need for paired audio-MIDI training data. Our primary contribution is a semi-supervised method to automatically curate a large and diverse corpus of one-shot drum samples from unlabeled audio sources. We then use this corpus to synthesize a high-quality dataset from MIDI files alone, which we use to train a sequence-to-sequence transcription model. We evaluate our model on the ENST and MDB test sets, where it achieves new state-of-the-art results, significantly outperforming both fully supervised methods and previous synthetic-data approaches. The code for reproducing our experiments is publicly available at https://github.com/pier-maker92/ADT_STR
Paper Structure (15 sections, 9 equations, 1 figure, 2 tables, 1 algorithm)

This paper contains 15 sections, 9 equations, 1 figure, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Methodology
  4. Instrument Vocabulary
  5. Semi-Supervised One-Shot Library Creation
  6. Tokenization and model architecture
  7. Tokenization
  8. Model architecture
  9. Experimental Evaluation
  10. Data Synthetic Generation
  11. Results
  12. Evaluation data
  13. Experimental settings
  14. Conclusion
  15. Acknowledgments

Figures (1)

  • Figure 1: Metrics reported for the 3-split cross-validation on ENST and MDB dataset.