Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RMVPE: A Robust Model for Vocal Pitch Estimation in Polyphonic Music

Haojie Wei, Xueke Cao, Tangpeng Dan, Yueguo Chen

TL;DR

A robust model named RMVPE is proposed that can extract effective hidden features and accurately predict vocal pitches from polyphonic music and is robust across all signal-to-noise ratio (SNR) levels.

Abstract

Vocal pitch is an important high-level feature in music audio processing. However, extracting vocal pitch in polyphonic music is more challenging due to the presence of accompaniment. To eliminate the influence of the accompaniment, most previous methods adopt music source separation models to obtain clean vocals from polyphonic music before predicting vocal pitches. As a result, the performance of vocal pitch estimation is affected by the music source separation models. To address this issue and directly extract vocal pitches from polyphonic music, we propose a robust model named RMVPE. This model can extract effective hidden features and accurately predict vocal pitches from polyphonic music. The experimental results demonstrate the superiority of RMVPE in terms of raw pitch accuracy (RPA) and raw chroma accuracy (RCA). Additionally, experiments conducted with different types of noise show that RMVPE is robust across all signal-to-noise ratio (SNR) levels. The code of RMVPE is available at https://github.com/Dream-High/RMVPE.

RMVPE: A Robust Model for Vocal Pitch Estimation in Polyphonic Music

TL;DR

A robust model named RMVPE is proposed that can extract effective hidden features and accurately predict vocal pitches from polyphonic music and is robust across all signal-to-noise ratio (SNR) levels.

Abstract

Vocal pitch is an important high-level feature in music audio processing. However, extracting vocal pitch in polyphonic music is more challenging due to the presence of accompaniment. To eliminate the influence of the accompaniment, most previous methods adopt music source separation models to obtain clean vocals from polyphonic music before predicting vocal pitches. As a result, the performance of vocal pitch estimation is affected by the music source separation models. To address this issue and directly extract vocal pitches from polyphonic music, we propose a robust model named RMVPE. This model can extract effective hidden features and accurately predict vocal pitches from polyphonic music. The experimental results demonstrate the superiority of RMVPE in terms of raw pitch accuracy (RPA) and raw chroma accuracy (RCA). Additionally, experiments conducted with different types of noise show that RMVPE is robust across all signal-to-noise ratio (SNR) levels. The code of RMVPE is available at https://github.com/Dream-High/RMVPE.
Paper Structure (13 sections, 4 equations, 3 figures, 3 tables)

This paper contains 13 sections, 4 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Model Architecture
  4. The Loss Function
  5. Experiment
  6. Datasets and Evaluation Metrics
  7. Experimental setup
  8. Results on Polyphonic Music
  9. Results with Different Types of Noise
  10. Noise Robustness
  11. Results on Clean Vocals
  12. Conclusion
  13. Acknowledgements

Figures (3)

  • Figure 1: The overall structure of our proposed model RMVPE.
  • Figure 2: (a) Residual Encoder Block (REB), (b) Residual Convolutional Block (RCB), (c) Residual Decoder Block (RDB)
  • Figure 3: The RPA (%) performance under different noise levels.