Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NV-Bench: Benchmark of Nonverbal Vocalization Synthesis for Expressive Text-to-Speech Generation

Qinke Ni, Huan Liao, Dekun Chen, Yuxiang Wang, Zhizheng Wu

Abstract

While recent text-to-speech (TTS) systems increasingly integrate nonverbal vocalizations (NVs), their evaluations lack standardized metrics and reliable ground-truth references. To bridge this gap, we propose NV-Bench, the first benchmark grounded in a functional taxonomy that treats NVs as communicative acts rather than acoustic artifacts. NV-Bench comprises 1,651 multi-lingual, in-the-wild utterances with paired human reference audio, balanced across 14 NV categories. We introduce a dual-dimensional evaluation protocol: (1) Instruction Alignment, utilizing the proposed paralinguistic character error rate (PCER) to assess controllability, (2) Acoustic Fidelity, measuring the distributional gap to real recordings to assess acoustic realism. We evaluate diverse TTS models and develop two baselines. Experimental results demonstrate a strong correlation between our objective metrics and human perception, establishing NV-Bench as a standardized evaluation framework.

NV-Bench: Benchmark of Nonverbal Vocalization Synthesis for Expressive Text-to-Speech Generation

Abstract

While recent text-to-speech (TTS) systems increasingly integrate nonverbal vocalizations (NVs), their evaluations lack standardized metrics and reliable ground-truth references. To bridge this gap, we propose NV-Bench, the first benchmark grounded in a functional taxonomy that treats NVs as communicative acts rather than acoustic artifacts. NV-Bench comprises 1,651 multi-lingual, in-the-wild utterances with paired human reference audio, balanced across 14 NV categories. We introduce a dual-dimensional evaluation protocol: (1) Instruction Alignment, utilizing the proposed paralinguistic character error rate (PCER) to assess controllability, (2) Acoustic Fidelity, measuring the distributional gap to real recordings to assess acoustic realism. We evaluate diverse TTS models and develop two baselines. Experimental results demonstrate a strong correlation between our objective metrics and human perception, establishing NV-Bench as a standardized evaluation framework.
Paper Structure (21 sections, 2 equations, 1 figure, 5 tables)

This paper contains 21 sections, 2 equations, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Multi-lingual NVASR
  4. Model architecture
  5. Data construction and label normalization
  6. Benchmark dataset construction
  7. Data collection
  8. Filtering pipeline
  9. NV-Bench
  10. Experiments
  11. Experiments for multi-lingual NVASR
  12. Experimental setup and baselines
  13. Evaluation metrics
  14. Results
  15. Benchmarking NV-Capable TTS models
  16. ...and 6 more sections

Figures (1)

  • Figure 1: Overview of the NV-Bench. (1) Data Processing: Raw audio is filtered using the Emilia-Pipeline and MiMo-Audio. (2) Multi-lingual NVASR: We train a multi-lingual NVASR model on open-source data with a unified label taxonomy. (3) Evaluation: After human verification, the benchmark is evaluated in instruction alignment and acoustic fidelity dimensions and human subjective ratings.