Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

StyleBench: Evaluating Speech Language Models on Conversational Speaking Style Control

Haishu Zhao, Aokai Hao, Yuan Ge, Zhenqiang Hong, Tong Xiao, Jingbo Zhu

Abstract

Speech language models (SLMs) have significantly extended the interactive capability of text-based Large Language Models (LLMs) by incorporating paralinguistic information. For more realistic interactive experience with customized styles, current SLMs have managed to interpret and control speaking style intensity from user prompts during the dialogue process. However, there remains a lack of systematic benchmarks that quantifies and evaluates the style intensity control ability in conversations. In this paper, we propose StyleBench, a multi-turn dialogue benchmark for comprehensively evaluating the style intensity control ability across four dimensions: emotion, speed, volume, and pitch. Our results reveal the performance gaps between leading SLMs and omni language models (OLMs), suggesting the underlying reasons and promising approaches for future exploration.

StyleBench: Evaluating Speech Language Models on Conversational Speaking Style Control

Abstract

Speech language models (SLMs) have significantly extended the interactive capability of text-based Large Language Models (LLMs) by incorporating paralinguistic information. For more realistic interactive experience with customized styles, current SLMs have managed to interpret and control speaking style intensity from user prompts during the dialogue process. However, there remains a lack of systematic benchmarks that quantifies and evaluates the style intensity control ability in conversations. In this paper, we propose StyleBench, a multi-turn dialogue benchmark for comprehensively evaluating the style intensity control ability across four dimensions: emotion, speed, volume, and pitch. Our results reveal the performance gaps between leading SLMs and omni language models (OLMs), suggesting the underlying reasons and promising approaches for future exploration.
Paper Structure (13 sections, 1 equation, 2 figures, 4 tables)

This paper contains 13 sections, 1 equation, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Benchmark Dataset Construction
  3. Data Samples
  4. Textual Content
  5. Speech Synthesis
  6. Evaluations & Discussions
  7. Style Quantification Metrics
  8. Experimental Setup
  9. Main Results
  10. Discussions
  11. Impact of Data Training
  12. Impact of Speech Tokenizers
  13. Conclusion

Figures (2)

  • Figure 1: Conversational Speaking Style Controlling Dialogue
  • Figure 2: An overview of data composition and synthesis process: textual contents are generated with different situational features (emotional or neutral), followed by neutral prompts and stylistic responses synthesized using CosyVoice2. Emotional responses were synthesized using the RAVDESS as the reference audio, while the others were adjusted using FFmpeg.