Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Comparative Evaluation of Expressive Japanese Character Text-to-Speech with VITS and Style-BERT-VITS2

Zackary Rackauckas, Julia Hirschberg

TL;DR

This work provides a first structured, per-character comparison of VITS and Style-BERT-VITS2 JP Extra (SBV2JE) for expressive Japanese character speech. It evaluates naturalness, CMOS, consistency, and intelligibility across three character datasets, showing SBV2JE closely matches ground truth MOS and achieves lower WER with a slight CMOS advantage. Architectural refinements in SBV2JE, including a WavLM-based discriminator and pitch-accent adjustments, underpin its performance gains, albeit with higher computational demands. The results highlight SBV2JE's suitability for character-driven language learning and dialogue applications while outlining avenues for broader generalization and deployment efficiency.

Abstract

Synthesizing expressive Japanese character speech poses unique challenges due to pitch-accent sensitivity and stylistic variability. This paper empirically evaluates two open-source text-to-speech models--VITS and Style-BERT-VITS2 JP Extra (SBV2JE)--on in-domain, character-driven Japanese speech. Using three character-specific datasets, we evaluate models across naturalness (mean opinion and comparative mean opinion score), intelligibility (word error rate), and speaker consistency. SBV2JE matches human ground truth in naturalness (MOS 4.37 vs. 4.38), achieves lower WER, and shows slight preference in CMOS. Enhanced by pitch-accent controls and a WavLM-based discriminator, SBV2JE proves effective for applications like language learning and character dialogue generation, despite higher computational demands.

Comparative Evaluation of Expressive Japanese Character Text-to-Speech with VITS and Style-BERT-VITS2

TL;DR

This work provides a first structured, per-character comparison of VITS and Style-BERT-VITS2 JP Extra (SBV2JE) for expressive Japanese character speech. It evaluates naturalness, CMOS, consistency, and intelligibility across three character datasets, showing SBV2JE closely matches ground truth MOS and achieves lower WER with a slight CMOS advantage. Architectural refinements in SBV2JE, including a WavLM-based discriminator and pitch-accent adjustments, underpin its performance gains, albeit with higher computational demands. The results highlight SBV2JE's suitability for character-driven language learning and dialogue applications while outlining avenues for broader generalization and deployment efficiency.

Abstract

Synthesizing expressive Japanese character speech poses unique challenges due to pitch-accent sensitivity and stylistic variability. This paper empirically evaluates two open-source text-to-speech models--VITS and Style-BERT-VITS2 JP Extra (SBV2JE)--on in-domain, character-driven Japanese speech. Using three character-specific datasets, we evaluate models across naturalness (mean opinion and comparative mean opinion score), intelligibility (word error rate), and speaker consistency. SBV2JE matches human ground truth in naturalness (MOS 4.37 vs. 4.38), achieves lower WER, and shows slight preference in CMOS. Enhanced by pitch-accent controls and a WavLM-based discriminator, SBV2JE proves effective for applications like language learning and character dialogue generation, despite higher computational demands.

Paper Structure

This paper contains 13 sections, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Models
  4. VITS
  5. Style-BERT-VITS2 JP Extra
  6. Method
  7. Experiments
  8. Mean Opinion Score
  9. Comparative Mean Opinion Score
  10. Intra-Speaker Consistency
  11. Word Error Rate
  12. Discussion
  13. Conclusion

Figures (1)

  • Figure 1: Pitch (F0) contours of self-introduction utterances (“X wa Y desu”) for the three characters. Hikari exhibits a rising contour followed by a sharp fall, Saya shows a mid-level onset with a steep drop and subsequent rise–fall, and Kitsune presents a gradual decline ending in a sharp final drop. These differences highlight the distinct prosodic patterns captured by each character’s voice.