Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Synthetic Singers: A Review of Deep-Learning-based Singing Voice Synthesis Approaches

Changhao Pan, Dongyu Yao, Yu Zhang, Wenxiang Guo, Jingyu Lu, Zhiyuan Zhu, Zhou Zhao

TL;DR

This survey addresses a central problem in music technology: generating high‑fidelity, controllable singing voices from textual and musical inputs. It surveys deep‑learning based SVS approaches by organizing them into cascaded and end‑to‑end architectures, and analyzes core technologies in singing modeling, control, and training strategies, along with datasets, annotation tools, and evaluation benchmarks. Key contributions include an up‑to‑date taxonomy of tasks (Hi‑Fi synthesis, controllable synthesis, style transfer, and text‑to‑song generation), a synthesis of representations (content, acoustic, semantic) and control mechanisms, and a comprehensive resource guide for datasets and evaluation methodologies. The paper also discusses training/inference strategies, future directions (e.g., multi‑modal conditioning, reinforcement learning, and MLLMs integration), and ethical considerations for data licensing and misuse prevention, making it a useful reference for researchers and practitioners aiming to advance SVS development.

Abstract

Recent advances in singing voice synthesis (SVS) have attracted substantial attention from both academia and industry. With the advent of large language models and novel generative paradigms, producing controllable, high-fidelity singing voices has become an attainable goal. Yet the field still lacks a comprehensive survey that systematically analyzes deep-learning-based singing voice synthesis systems and their enabling technologies. To address the aforementioned issue, this survey first categorizes existing systems by task type and then organizes current architectures into two major paradigms: cascaded and end-to-end approaches. Moreover, we provide an in-depth analysis of core technologies, covering singing modeling and control techniques. Finally, we review relevant datasets, annotation tools, and evaluation benchmarks that support training and assessment. In appendix, we introduce training strategies and further discussion of SVS. This survey provides an up-to-date review of the literature on SVS models, which would be a useful reference for both researchers and engineers. Related materials are available at https://github.com/David-Pigeon/SyntheticSingers.

Synthetic Singers: A Review of Deep-Learning-based Singing Voice Synthesis Approaches

TL;DR

This survey addresses a central problem in music technology: generating high‑fidelity, controllable singing voices from textual and musical inputs. It surveys deep‑learning based SVS approaches by organizing them into cascaded and end‑to‑end architectures, and analyzes core technologies in singing modeling, control, and training strategies, along with datasets, annotation tools, and evaluation benchmarks. Key contributions include an up‑to‑date taxonomy of tasks (Hi‑Fi synthesis, controllable synthesis, style transfer, and text‑to‑song generation), a synthesis of representations (content, acoustic, semantic) and control mechanisms, and a comprehensive resource guide for datasets and evaluation methodologies. The paper also discusses training/inference strategies, future directions (e.g., multi‑modal conditioning, reinforcement learning, and MLLMs integration), and ethical considerations for data licensing and misuse prevention, making it a useful reference for researchers and practitioners aiming to advance SVS development.

Abstract

Recent advances in singing voice synthesis (SVS) have attracted substantial attention from both academia and industry. With the advent of large language models and novel generative paradigms, producing controllable, high-fidelity singing voices has become an attainable goal. Yet the field still lacks a comprehensive survey that systematically analyzes deep-learning-based singing voice synthesis systems and their enabling technologies. To address the aforementioned issue, this survey first categorizes existing systems by task type and then organizes current architectures into two major paradigms: cascaded and end-to-end approaches. Moreover, we provide an in-depth analysis of core technologies, covering singing modeling and control techniques. Finally, we review relevant datasets, annotation tools, and evaluation benchmarks that support training and assessment. In appendix, we introduce training strategies and further discussion of SVS. This survey provides an up-to-date review of the literature on SVS models, which would be a useful reference for both researchers and engineers. Related materials are available at https://github.com/David-Pigeon/SyntheticSingers.
Paper Structure (51 sections, 3 figures, 1 table)

This paper contains 51 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Tasks of SVS
  3. Hi-Fidelity Synthesis
  4. Controllable Synthesis
  5. Singing Style Transfer
  6. Text-to-Song Generation
  7. Architecture
  8. Cascaded SVS Systems
  9. Acoustic Model
  10. Vocoder
  11. End-to-End SVS Systems
  12. SVS Representations and Control
  13. Representations and Modeling of SVS
  14. Singing's Content Representation
  15. Singing's Acoustic Representation
  16. ...and 36 more sections

Figures (3)

  • Figure 1: An overall demonstration of four prevailing tasks of SVS systems.
  • Figure 2: We categorize SVS models into two paradigms, cascaded and end-to-end approaches. A system is end-to-end if it outputs waveforms directly, without intermediate modules or hand-crafted interfaces. Thus requiring a vocoder implies a cascaded design. Dashed lines indicate optional process.
  • Figure 3: Three data annotation tasks required by the singing voice synthesis system.