Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantifying Speaker Embedding Phonological Rule Interactions in Accented Speech Synthesis

Thanathai Lertpetchpun, Yoonjeong Lee, Thanapat Trachu, Jihwan Lee, Tiantian Feng, Dani Byrd, Shrikanth Narayanan

TL;DR

This work investigates how linguistically motivated phonological rules interact with speaker embeddings in accented speech synthesis. By applying three well-documented American–British contrasts (flapping, rhoticity, and vowel correspondences) and introducing the phoneme shift rate $PSR = \frac{N2}{N1}$, the authors quantify the extent to which embeddings preserve or override rule-based transformations. Using Kokoro TTS, Vox-Profile accent classification, and UTMOS naturalness, they show that combining rules with embeddings yields more authentic accents while embeddings can dampen rule effects, revealing entanglement between accent and speaker identity. The results position rule-based interventions as practical, interpretable levers for accent control and provide a framework for assessing disentanglement in data-driven speech generation, with implications for more controllable and transparent TTS systems.

Abstract

Many spoken languages, including English, exhibit wide variation in dialects and accents, making accent control an important capability for flexible text-to-speech (TTS) models. Current TTS systems typically generate accented speech by conditioning on speaker embeddings associated with specific accents. While effective, this approach offers limited interpretability and controllability, as embeddings also encode traits such as timbre and emotion. In this study, we analyze the interaction between speaker embeddings and linguistically motivated phonological rules in accented speech synthesis. Using American and British English as a case study, we implement rules for flapping, rhoticity, and vowel correspondences. We propose the phoneme shift rate (PSR), a novel metric quantifying how strongly embeddings preserve or override rule-based transformations. Experiments show that combining rules with embeddings yields more authentic accents, while embeddings can attenuate or overwrite rules, revealing entanglement between accent and speaker identity. Our findings highlight rules as a lever for accent control and a framework for evaluating disentanglement in speech generation.

Quantifying Speaker Embedding Phonological Rule Interactions in Accented Speech Synthesis

TL;DR

This work investigates how linguistically motivated phonological rules interact with speaker embeddings in accented speech synthesis. By applying three well-documented American–British contrasts (flapping, rhoticity, and vowel correspondences) and introducing the phoneme shift rate , the authors quantify the extent to which embeddings preserve or override rule-based transformations. Using Kokoro TTS, Vox-Profile accent classification, and UTMOS naturalness, they show that combining rules with embeddings yields more authentic accents while embeddings can dampen rule effects, revealing entanglement between accent and speaker identity. The results position rule-based interventions as practical, interpretable levers for accent control and provide a framework for assessing disentanglement in data-driven speech generation, with implications for more controllable and transparent TTS systems.

Abstract

Many spoken languages, including English, exhibit wide variation in dialects and accents, making accent control an important capability for flexible text-to-speech (TTS) models. Current TTS systems typically generate accented speech by conditioning on speaker embeddings associated with specific accents. While effective, this approach offers limited interpretability and controllability, as embeddings also encode traits such as timbre and emotion. In this study, we analyze the interaction between speaker embeddings and linguistically motivated phonological rules in accented speech synthesis. Using American and British English as a case study, we implement rules for flapping, rhoticity, and vowel correspondences. We propose the phoneme shift rate (PSR), a novel metric quantifying how strongly embeddings preserve or override rule-based transformations. Experiments show that combining rules with embeddings yields more authentic accents, while embeddings can attenuate or overwrite rules, revealing entanglement between accent and speaker identity. Our findings highlight rules as a lever for accent control and a framework for evaluating disentanglement in speech generation.
Paper Structure (15 sections, 2 figures, 3 tables)

This paper contains 15 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Phonological Rules
  3. Application to Speech Generation Tasks
  4. Metrics
  5. Vox-Profile Accent Classifier
  6. Phoneme Shift Rate (PSR)
  7. Naturalness Evaluation with UTMOS
  8. Datasets and Experimental Setups
  9. Rule Application Configurations
  10. Results and Discussion
  11. Effects of Each Phonological Rule on Speech Synthesis
  12. How Do Rules Affect Phonemes of the Synthesized Speech?
  13. On the Role of Individual Speaker Embeddings in Accented Speech Synthesis.
  14. Conclusion
  15. Acknowledgment

Figures (2)

  • Figure 1: Synthesis and evaluation pipeline. Inputs to Kokoro TTS are a phoneme sequence (American or British), a speaker embedding, and fixed phoneme durations. The yellow path shows rule-based phoneme substitutions prior to synthesis, and the green path shows evaluation of their effect on the synthesized accented speech.
  • Figure 2: Kernel Density Estimation (KDE) of histograms on the number of changes of the rules.