Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating and Rewarding LALMs for Expressive Role-Play TTS via Mean Continuation Log-Probability

Yong Ren, Jingbei Li, Haiyang Sun, Yujie Chen, Cheng Yi, Yechang Huang, Hao Gu, Ye Bai, Xuerui Yang

TL;DR

The paper tackles the challenge of maintaining stylistic coherence in expressive Role-Play TTS (RP-TTS) by introducing Mean Continuation Log-Probability (MCLP), a continuation-based metric derived from pre-trained Large Audio Language Models to quantify stylistic consistency. It leverages In-Context Learning priors to compute MCLP as both an objective evaluation and a reinforcement learning reward, integrated into a GRPO framework with a hybrid reward that also penalizes content misalignment via CER. A large RP-TTS dataset with rich scene and character annotations is constructed to enable robust evaluation, with SFT used to instill scene/character constraints and RL to enhance multi-turn style alignment. Empirical results show that the proposed approach surpasses strong LALM baselines on both objective (CER, Pinyin WER, MCLP) and subjective (MOS) metrics, and ablations reveal the necessity of combining SFT and RL while highlighting the risks of reward hacking when optimizing a single objective. The work provides both a practical metric for stylistic evaluation and a principled training signal for achieving coherent, scene-faithful RP-TTS in multi-turn dialogues, with implications for more controllable and engaging speech systems.

Abstract

Recent advances in Large Audio Language Models (LALMs) have extended Text-to-Speech (TTS) to interactive role-play scenarios, which demand high expressiveness and strict adherence to role-play instructions. However, existing models struggle to maintain stylistic consistency with character profiles and scene descriptions across multi-turn dialogues. A critical bottleneck is the lack of objective metrics for quantifying speaking style. To bridge this gap, we propose Mean Continuation Log-Probability (MCLP) as both an evaluation metric and a reward signal, validated on LALM-based Role-Play TTS (RP-TTS) tasks. Critically, we leverage the In-Context Learning capability of pre-trained LALMs to formulate MCLP via a continuation log-probability prediction. This metric quantifies stylistic consistency by measuring the likelihood of the ground-truth speech conditioned on the generated speech. Furthermore, we employ MCLP as a reinforcement learning reward to enhance the style alignment between generated speech and Role-Play instructions. To facilitate evaluation, we construct an RP-TTS dataset with rich scene and character annotations. Experimental results demonstrate that our method significantly outperforms strong LALM baselines on both objective and subjective metrics.

Evaluating and Rewarding LALMs for Expressive Role-Play TTS via Mean Continuation Log-Probability

TL;DR

The paper tackles the challenge of maintaining stylistic coherence in expressive Role-Play TTS (RP-TTS) by introducing Mean Continuation Log-Probability (MCLP), a continuation-based metric derived from pre-trained Large Audio Language Models to quantify stylistic consistency. It leverages In-Context Learning priors to compute MCLP as both an objective evaluation and a reinforcement learning reward, integrated into a GRPO framework with a hybrid reward that also penalizes content misalignment via CER. A large RP-TTS dataset with rich scene and character annotations is constructed to enable robust evaluation, with SFT used to instill scene/character constraints and RL to enhance multi-turn style alignment. Empirical results show that the proposed approach surpasses strong LALM baselines on both objective (CER, Pinyin WER, MCLP) and subjective (MOS) metrics, and ablations reveal the necessity of combining SFT and RL while highlighting the risks of reward hacking when optimizing a single objective. The work provides both a practical metric for stylistic evaluation and a principled training signal for achieving coherent, scene-faithful RP-TTS in multi-turn dialogues, with implications for more controllable and engaging speech systems.

Abstract

Recent advances in Large Audio Language Models (LALMs) have extended Text-to-Speech (TTS) to interactive role-play scenarios, which demand high expressiveness and strict adherence to role-play instructions. However, existing models struggle to maintain stylistic consistency with character profiles and scene descriptions across multi-turn dialogues. A critical bottleneck is the lack of objective metrics for quantifying speaking style. To bridge this gap, we propose Mean Continuation Log-Probability (MCLP) as both an evaluation metric and a reward signal, validated on LALM-based Role-Play TTS (RP-TTS) tasks. Critically, we leverage the In-Context Learning capability of pre-trained LALMs to formulate MCLP via a continuation log-probability prediction. This metric quantifies stylistic consistency by measuring the likelihood of the ground-truth speech conditioned on the generated speech. Furthermore, we employ MCLP as a reinforcement learning reward to enhance the style alignment between generated speech and Role-Play instructions. To facilitate evaluation, we construct an RP-TTS dataset with rich scene and character annotations. Experimental results demonstrate that our method significantly outperforms strong LALM baselines on both objective and subjective metrics.
Paper Structure (38 sections, 6 equations, 5 figures, 3 tables)

This paper contains 38 sections, 6 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM-Based Controllable TTS
  4. Speech Role-Playing Agents
  5. RL for TTS Alignment
  6. Problem Preliminaries
  7. Methodology
  8. Continuation Score
  9. Definition of MCLP.
  10. Information-Theoretic Analysis.
  11. Supervised Fine-Tuning
  12. Reinforcement Learning
  13. Hybrid Reward Formulation.
  14. Dataset Construction
  15. Data Acquisition and Preprocessing
  16. ...and 23 more sections

Figures (5)

  • Figure 1: The overall framework of our proposed method. (a) The pretraining stage of the LALM for computing MCLP. (b) The LALM is fine-tuned to generate interleaved TA4 tokens conditioned on a structured prompt containing scene descriptions, character profiles, and dialogue history. (c) The hybrid reward function for GRPO that synergizes the style-centric MCLP signal with a content-fidelity penalty (CER) to prevent reward hacking.
  • Figure 2: An example for Role-Play TTS.
  • Figure 3: Data Curation and Annotation Pipeline. We construct the RP-TTS dataset from the WenetSpeech corpus via a multi-stage process.
  • Figure 4: Distribution of dialogue turns. The number of utterances per scene in our RP-TTS dataset.
  • Figure 5: Win rate vs. $\Delta$MCLP Score. The win rate (probability that a higher MCLP score predicts a higher human MOS) across different $\Delta$MCLP bins.