Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Which Prosodic Features Matter Most for Pragmatics?

Nigel G. Ward, Divette Marco, Olac Fuentes

TL;DR

This paper asks which prosodic features best support pragmatic functions in dialog by predicting human judgments of pragmatic similarity between utterance pairs. It evaluates a broad set of engineered prosodic features, including duration-related measures and pitch metrics, against a range of models from simple distance measures to Random Forests and a pretrained HuBERT representation, using English and Spanish data. The key finding is that duration-related features (especially speaking rate) are more informative than pitch features, while pitch alone often fails to capture important pragmatic cues; nasality and vibrato emerge as signals overlooked by typical feature sets. The work has practical implications for speech-synthesis evaluation, loss-function design, and the selection of prosodic features in pragmatic modeling, though it is limited by data size, feature simplicity, and language scope, suggesting avenues for broader cross-linguistic studies and richer representations.

Abstract

We investigate which prosodic features matter most in conveying prosodic functions. We use the problem of predicting human perceptions of pragmatic similarity among utterance pairs to evaluate the utility of prosodic features of different types. We find, for example, that duration-related features are more important than pitch-related features, and that utterance-initial features are more important than utterance-final features. Further, failure analysis indicates that modeling using pitch features only often fails to handle important pragmatic functions, and suggests that several generally-neglected acoustic and prosodic features are pragmatically significant, including nasality and vibrato. These findings can guide future basic research in prosody, and suggest how to improve speech synthesis evaluation, among other applications.

Which Prosodic Features Matter Most for Pragmatics?

TL;DR

This paper asks which prosodic features best support pragmatic functions in dialog by predicting human judgments of pragmatic similarity between utterance pairs. It evaluates a broad set of engineered prosodic features, including duration-related measures and pitch metrics, against a range of models from simple distance measures to Random Forests and a pretrained HuBERT representation, using English and Spanish data. The key finding is that duration-related features (especially speaking rate) are more informative than pitch features, while pitch alone often fails to capture important pragmatic cues; nasality and vibrato emerge as signals overlooked by typical feature sets. The work has practical implications for speech-synthesis evaluation, loss-function design, and the selection of prosodic features in pragmatic modeling, though it is limited by data size, feature simplicity, and language scope, suggesting avenues for broader cross-linguistic studies and richer representations.

Abstract

We investigate which prosodic features matter most in conveying prosodic functions. We use the problem of predicting human perceptions of pragmatic similarity among utterance pairs to evaluate the utility of prosodic features of different types. We find, for example, that duration-related features are more important than pitch-related features, and that utterance-initial features are more important than utterance-final features. Further, failure analysis indicates that modeling using pitch features only often fails to handle important pragmatic functions, and suggests that several generally-neglected acoustic and prosodic features are pragmatically significant, including nasality and vibrato. These findings can guide future basic research in prosody, and suggest how to improve speech synthesis evaluation, among other applications.
Paper Structure (11 sections, 2 figures, 2 tables)

This paper contains 11 sections, 2 figures, 2 tables.

Table of Contents

  1. Motivation
  2. Methods
  3. Data
  4. Features
  5. Models and Prediction Results
  6. Feature-Importance Analyses
  7. Most Important Feature Types
  8. Most Important Feature Positions
  9. Qualitative Analyses
  10. Spanish
  11. Summary and Limitations

Figures (2)

  • Figure 1: Single-feature correlations between the judgments and the deltas for five of the most informative feature types, across both Session 1 and Session 2 data. The X-axis represents the regions, defined by fixed percentages of the utterance duration.
  • Figure 2: Feature importance as a function of position (time slice): on the left axis, performance of a model using only features at that position; on the right axis, summed importance in the random forest regression model.