Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LHQ-SVC: Lightweight and High Quality Singing Voice Conversion Modeling

Yubo Huang, Xin Lai, Muyang Ye, Anran Zhu, Zixi Wang, Jingzehua Xu, Shuai Zhang, Zhiyuan Zhou, Weijie Niu

TL;DR

This work tackles the problem of high-quality Singing Voice Conversion with limited computational resources by introducing LHQ-SVC, a lightweight diffusion-based model optimized for CPU execution. The approach combines model-structure optimization, dynamic module adjustment, and sampling distillation to preserve timbre and melody while reducing latency and memory footprint. Experiments on the SVC-2023 and DAMP datasets show that LHQ-SVC achieves competitive PESQ and MOS scores at model sizes around 68 MB, with a mobile variant at about 53 MB, and substantial speedups on multi-core CPUs and mobile hardware. The results demonstrate the practicality of real-time or near-real-time SVC on resource-constrained devices and point to future work in broader language support and further efficiency gains.

Abstract

Singing Voice Conversion (SVC) has emerged as a significant subfield of Voice Conversion (VC), enabling the transformation of one singer's voice into another while preserving musical elements such as melody, rhythm, and timbre. Traditional SVC methods have limitations in terms of audio quality, data requirements, and computational complexity. In this paper, we propose LHQ-SVC, a lightweight, CPU-compatible model based on the SVC framework and diffusion model, designed to reduce model size and computational demand without sacrificing performance. We incorporate features to improve inference quality, and optimize for CPU execution by using performance tuning tools and parallel computing frameworks. Our experiments demonstrate that LHQ-SVC maintains competitive performance, with significant improvements in processing speed and efficiency across different devices. The results suggest that LHQ-SVC can meet

LHQ-SVC: Lightweight and High Quality Singing Voice Conversion Modeling

TL;DR

This work tackles the problem of high-quality Singing Voice Conversion with limited computational resources by introducing LHQ-SVC, a lightweight diffusion-based model optimized for CPU execution. The approach combines model-structure optimization, dynamic module adjustment, and sampling distillation to preserve timbre and melody while reducing latency and memory footprint. Experiments on the SVC-2023 and DAMP datasets show that LHQ-SVC achieves competitive PESQ and MOS scores at model sizes around 68 MB, with a mobile variant at about 53 MB, and substantial speedups on multi-core CPUs and mobile hardware. The results demonstrate the practicality of real-time or near-real-time SVC on resource-constrained devices and point to future work in broader language support and further efficiency gains.

Abstract

Singing Voice Conversion (SVC) has emerged as a significant subfield of Voice Conversion (VC), enabling the transformation of one singer's voice into another while preserving musical elements such as melody, rhythm, and timbre. Traditional SVC methods have limitations in terms of audio quality, data requirements, and computational complexity. In this paper, we propose LHQ-SVC, a lightweight, CPU-compatible model based on the SVC framework and diffusion model, designed to reduce model size and computational demand without sacrificing performance. We incorporate features to improve inference quality, and optimize for CPU execution by using performance tuning tools and parallel computing frameworks. Our experiments demonstrate that LHQ-SVC maintains competitive performance, with significant improvements in processing speed and efficiency across different devices. The results suggest that LHQ-SVC can meet
Paper Structure (12 sections, 7 equations, 2 figures, 1 table)

This paper contains 12 sections, 7 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Voice Conversion
  4. Diffusion Model Applications
  5. Methodology
  6. Model structure optimization and dynamic adjustment
  7. Sampling mechanism optimization and efficient reasoning.
  8. Experiments
  9. Dataset Preparation
  10. Training Process
  11. Experimental Models and Results
  12. Conclusions

Figures (2)

  • Figure 1: Progressive distillation diagram for diffusion modelling
  • Figure 2: Comparison of GPU and CPU inference time on computer and cell phones