Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

S$^2$Voice: Style-Aware Autoregressive Modeling with Enhanced Conditioning for Singing Style Conversion

Ziqian Wang, Xianjun Xia, Chuanzeng Huang, Lei Xie

TL;DR

S$^2$Voice tackles singing style conversion by extending a two-stage Vevo baseline with explicit style control and robust timbre preservation. The AR LLM stage employs FiLM-style layer normalization and a style-aware cross-attention mechanism to encode fine-grained style from a compact embedding sequence, while a global speaker embedding guides the flow-matching decoder to maintain timbre in the acoustic stage. A large-scale curated singing corpus and a multi-stage training regime (SFT followed by DPO) enhance perceptual quality, stability, and zero-shot generalization. Experimental results on SVCC 2025 demonstrate state-of-the-art performance across in-domain and zero-shot tasks, with ablations confirming the complementary benefits of style conditioning, timbre guidance, and data-centric training.

Abstract

We present S$^2$Voice, the winning system of the Singing Voice Conversion Challenge (SVCC) 2025 for both the in-domain and zero-shot singing style conversion tracks. Built on the strong two-stage Vevo baseline, S$^2$Voice advances style control and robustness through several contributions. First, we integrate style embeddings into the autoregressive large language model (AR LLM) via a FiLM-style layer-norm conditioning and a style-aware cross-attention for enhanced fine-grained style modeling. Second, we introduce a global speaker embedding into the flow-matching transformer to improve timbre similarity. Third, we curate a large, high-quality singing corpus via an automated pipeline for web harvesting, vocal separation, and transcript refinement. Finally, we employ a multi-stage training strategy combining supervised fine-tuning (SFT) and direct preference optimization (DPO). Subjective listening tests confirm our system's superior performance: leading in style similarity and singer similarity for Task 1, and across naturalness, style similarity, and singer similarity for Task 2. Ablation studies demonstrate the effectiveness of our contributions in enhancing style fidelity, timbre preservation, and generalization. Audio samples are available~\footnote{https://honee-w.github.io/SVC-Challenge-Demo/}.

S$^2$Voice: Style-Aware Autoregressive Modeling with Enhanced Conditioning for Singing Style Conversion

TL;DR

SVoice tackles singing style conversion by extending a two-stage Vevo baseline with explicit style control and robust timbre preservation. The AR LLM stage employs FiLM-style layer normalization and a style-aware cross-attention mechanism to encode fine-grained style from a compact embedding sequence, while a global speaker embedding guides the flow-matching decoder to maintain timbre in the acoustic stage. A large-scale curated singing corpus and a multi-stage training regime (SFT followed by DPO) enhance perceptual quality, stability, and zero-shot generalization. Experimental results on SVCC 2025 demonstrate state-of-the-art performance across in-domain and zero-shot tasks, with ablations confirming the complementary benefits of style conditioning, timbre guidance, and data-centric training.

Abstract

We present SVoice, the winning system of the Singing Voice Conversion Challenge (SVCC) 2025 for both the in-domain and zero-shot singing style conversion tracks. Built on the strong two-stage Vevo baseline, SVoice advances style control and robustness through several contributions. First, we integrate style embeddings into the autoregressive large language model (AR LLM) via a FiLM-style layer-norm conditioning and a style-aware cross-attention for enhanced fine-grained style modeling. Second, we introduce a global speaker embedding into the flow-matching transformer to improve timbre similarity. Third, we curate a large, high-quality singing corpus via an automated pipeline for web harvesting, vocal separation, and transcript refinement. Finally, we employ a multi-stage training strategy combining supervised fine-tuning (SFT) and direct preference optimization (DPO). Subjective listening tests confirm our system's superior performance: leading in style similarity and singer similarity for Task 1, and across naturalness, style similarity, and singer similarity for Task 2. Ablation studies demonstrate the effectiveness of our contributions in enhancing style fidelity, timbre preservation, and generalization. Audio samples are available~\footnote{https://honee-w.github.io/SVC-Challenge-Demo/}.
Paper Structure (14 sections, 10 equations, 1 figure, 2 tables)

This paper contains 14 sections, 10 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Disentangled Speech Representation
  4. Two-stage Frameworks
  5. Method
  6. Overview
  7. Content--style Modeling with Style Conditioning
  8. Acoustic Modeling with Speaker Conditioning
  9. Data Pipeline and Multi-stage Training
  10. Experiments
  11. Datasets and Evaluation Metrics
  12. Experimental Setup
  13. Experimental Results
  14. Conclusion

Figures (1)

  • Figure 1: Autoregressive transformer block. (a) Original AR block with standard self-attention and feed-forward layers using conventional LayerNorm. (b) Modified AR block used in our AR-LLM: FiLM-style layer-norm modulation injects global style scale and shift ($\gamma,\beta$) produced by the style encoder, and a style-aware cross-attention lets local style queries attend to LLM context for fine-grained conditioning.