Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Data-Driven Analysis of Robust Automatic Piano Transcription

Drew Edwards, Simon Dixon, Emmanouil Benetos, Akira Maezawa, Yuta Kusaka

TL;DR

The paper addresses the robustness of automatic piano transcription by shifting focus from architecture to training data. It introduces Studio MAESTRO, a studio-recorded re-interpretation of MAESTRO, and a data augmentation pipeline that blends original, studio, and synthesized timbres to boost generalization. Through extensive experiments, it achieves a MAPS test $F1$ of 88.4 without MAPS training data, and demonstrates the critical role of pitch shifting and reverb in improving out-of-distribution performance. The findings highlight the importance of data diversity and out-of-distribution evaluation for reliable piano transcription systems, and suggest avenues for further timbral enrichment and robustness analysis.

Abstract

Algorithms for automatic piano transcription have improved dramatically in recent years due to new datasets and modeling techniques. Recent developments have focused primarily on adapting new neural network architectures, such as the Transformer and Perceiver, in order to yield more accurate systems. In this work, we study transcription systems from the perspective of their training data. By measuring their performance on out-of-distribution annotated piano data, we show how these models can severely overfit to acoustic properties of the training data. We create a new set of audio for the MAESTRO dataset, captured automatically in a professional studio recording environment via Yamaha Disklavier playback. Using various data augmentation techniques when training with the original and re-performed versions of the MAESTRO dataset, we achieve state-of-the-art note-onset accuracy of 88.4 F1-score on the MAPS dataset, without seeing any of its training data. We subsequently analyze these data augmentation techniques in a series of ablation studies to better understand their influence on the resulting models.

A Data-Driven Analysis of Robust Automatic Piano Transcription

TL;DR

The paper addresses the robustness of automatic piano transcription by shifting focus from architecture to training data. It introduces Studio MAESTRO, a studio-recorded re-interpretation of MAESTRO, and a data augmentation pipeline that blends original, studio, and synthesized timbres to boost generalization. Through extensive experiments, it achieves a MAPS test of 88.4 without MAPS training data, and demonstrates the critical role of pitch shifting and reverb in improving out-of-distribution performance. The findings highlight the importance of data diversity and out-of-distribution evaluation for reliable piano transcription systems, and suggest avenues for further timbral enrichment and robustness analysis.

Abstract

Algorithms for automatic piano transcription have improved dramatically in recent years due to new datasets and modeling techniques. Recent developments have focused primarily on adapting new neural network architectures, such as the Transformer and Perceiver, in order to yield more accurate systems. In this work, we study transcription systems from the perspective of their training data. By measuring their performance on out-of-distribution annotated piano data, we show how these models can severely overfit to acoustic properties of the training data. We create a new set of audio for the MAESTRO dataset, captured automatically in a professional studio recording environment via Yamaha Disklavier playback. Using various data augmentation techniques when training with the original and re-performed versions of the MAESTRO dataset, we achieve state-of-the-art note-onset accuracy of 88.4 F1-score on the MAPS dataset, without seeing any of its training data. We subsequently analyze these data augmentation techniques in a series of ablation studies to better understand their influence on the resulting models.
Paper Structure (12 sections, 2 figures, 3 tables)

This paper contains 12 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Robustness of Piano Transcription Systems
  4. Studio MAESTRO Dataset
  5. Training Experiments
  6. Data Augmentation Experiment
  7. Results
  8. Data Augmentation Analysis
  9. Data Degradation Study
  10. Single Augmentations
  11. Ablation Study
  12. Conclusion

Figures (2)

  • Figure 1: Illustration of the data augmentation pipeline employed. The process flows from left to right, starting with a Random 7-Band EQ, where each band's gain is randomly adjusted within a range of -10dB to 5dB. Next, random background noise is added, selected from a set of 65 different 10-second clips, with the signal-to-noise ratio meticulously maintained between 17.5 and 25dB. Then we apply a random pitch shift, varying between -10 and 10 cents, to account for slight tuning discrepancies in production data. Another Random 7-Band EQ is then applied. The final stage of the pipeline introduces reverb, utilizing one of 14 unique impulse responses, to simulate different spatial acoustic characteristics. Each component of this pipeline is applied independently with a probability of 0.5, ensuring a rich and diverse set of augmentations.
  • Figure 2: Data Degradation Results, showing the effect of data augmentation on performance. Kong et al.'s model is trained only on MAESTRO's training data. Our model benefits from a diversified training set, including original, studio, and synthesized MAESTRO recordings, all augmented. Evaluation spans test splits from MAESTRO, Studio MAESTRO, and MAPS, focusing on the note-onset F1 score.