Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DJCM: A Deep Joint Cascade Model for Singing Voice Separation and Vocal Pitch Estimation

Haojie Wei, Xueke Cao, Wenbo Xu, Tangpeng Dan, Yueguo Chen

TL;DR

The proposed Deep Joint Cascade Model (DJCM) employs a novel joint cascade model structure to concurrently train both tasks, and task-specific weights are used to align different objectives of both tasks.

Abstract

Singing voice separation and vocal pitch estimation are pivotal tasks in music information retrieval. Existing methods for simultaneous extraction of clean vocals and vocal pitches can be classified into two categories: pipeline methods and naive joint learning methods. However, the efficacy of these methods is limited by the following problems: On the one hand, pipeline methods train models for each task independently, resulting a mismatch between the data distributions at the training and testing time. On the other hand, naive joint learning methods simply add the losses of both tasks, possibly leading to a misalignment between the distinct objectives of each task. To solve these problems, we propose a Deep Joint Cascade Model (DJCM) for singing voice separation and vocal pitch estimation. DJCM employs a novel joint cascade model structure to concurrently train both tasks. Moreover, task-specific weights are used to align different objectives of both tasks. Experimental results show that DJCM achieves state-of-the-art performance on both tasks, with great improvements of 0.45 in terms of Signal-to-Distortion Ratio (SDR) for singing voice separation and 2.86% in terms of Overall Accuracy (OA) for vocal pitch estimation. Furthermore, extensive ablation studies validate the effectiveness of each design of our proposed model. The code of DJCM is available at https://github.com/Dream-High/DJCM .

DJCM: A Deep Joint Cascade Model for Singing Voice Separation and Vocal Pitch Estimation

TL;DR

The proposed Deep Joint Cascade Model (DJCM) employs a novel joint cascade model structure to concurrently train both tasks, and task-specific weights are used to align different objectives of both tasks.

Abstract

Singing voice separation and vocal pitch estimation are pivotal tasks in music information retrieval. Existing methods for simultaneous extraction of clean vocals and vocal pitches can be classified into two categories: pipeline methods and naive joint learning methods. However, the efficacy of these methods is limited by the following problems: On the one hand, pipeline methods train models for each task independently, resulting a mismatch between the data distributions at the training and testing time. On the other hand, naive joint learning methods simply add the losses of both tasks, possibly leading to a misalignment between the distinct objectives of each task. To solve these problems, we propose a Deep Joint Cascade Model (DJCM) for singing voice separation and vocal pitch estimation. DJCM employs a novel joint cascade model structure to concurrently train both tasks. Moreover, task-specific weights are used to align different objectives of both tasks. Experimental results show that DJCM achieves state-of-the-art performance on both tasks, with great improvements of 0.45 in terms of Signal-to-Distortion Ratio (SDR) for singing voice separation and 2.86% in terms of Overall Accuracy (OA) for vocal pitch estimation. Furthermore, extensive ablation studies validate the effectiveness of each design of our proposed model. The code of DJCM is available at https://github.com/Dream-High/DJCM .
Paper Structure (15 sections, 3 equations, 3 figures, 3 tables)

This paper contains 15 sections, 3 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Problem Formulation
  4. Model Architecture
  5. The Loss of Joint Learning
  6. Experimental Setup
  7. Datasets
  8. Evaluation Metrics
  9. Implementation Details
  10. Experimental Results
  11. Comparison With Previous Methods
  12. Ablation Study for Model Structure
  13. Ablation Study for Joint Learning
  14. Conclusion
  15. Acknowledgements

Figures (3)

  • Figure 1: The overall structure of our proposed model DJCM.
  • Figure 2: (a) Residual Encoder Block (REB), (b) Residual Convolutional Block (RCB), (c) Residual Decoder Block (RDB)
  • Figure 3: (a) The Shared Bottom structure. (b) The MMOE structure, the number of experts can be adjusted.