Conversational Feedback in Scripted versus Spontaneous Dialogues: A Comparative Analysis

TL;DR

It is shown that communicative feedback is markedly less frequent in subtitles than in spontaneous dialogues and subtitles contain a higher proportion of negative feedback.

Abstract

Scripted dialogues such as movie and TV subtitles constitute a widespread source of training data for conversational NLP models. However, there are notable linguistic differences between these dialogues and spontaneous interactions, especially regarding the occurrence of communicative feedback such as backchannels, acknowledgments, or clarification requests. This paper presents a quantitative analysis of such feedback phenomena in both subtitles and spontaneous conversations. Based on conversational data spanning eight languages and multiple genres, we extract lexical statistics, classifications from a dialogue act tagger, expert annotations and labels derived from a fine-tuned Large Language Model (LLM). Our main empirical findings are that (1) communicative feedback is markedly less frequent in subtitles than in spontaneous dialogues and (2) subtitles contain a higher proportion of negative feedback. We also show that dialogues generated by standard LLMs lie much closer to scripted dialogues than spontaneous interactions in terms of communicative feedback.

Figures (13)

• Figure 1: Frequency of conversational feedback of various types among utterances in the English corpora (both spontaneous and subtitles) based on manually curated lists of cue words to detect. Fig. (a) shows the absolute frequency while Fig. (b) zooms in on utterances labelled with at least one feedback. + denotes positive feedback/acknowledgement, = neutral/continuer feedback, -- negative feedback, ? clarification requests and ‘OTH’ is for other utterances. fo and hi respectively stand for ‘foreign audience’ and ‘hearing-impaired’ subtitles. Corpora without these prefixes are spontaneous dialogues.
• Figure 2: Most common lexical items associated with communicative feedback, as detected through manually curated lists of cue words in English, factored by corpus.
• Figure 3: Frequency of communicative feedback depending on the source of the dialogue sample (spontaneous interactions or subtitles) and the category of feedback, based on annotations from human experts.
• Figure 4: Frequency of communicative feedback depending on the corpus type and category of feedback, based on the predictions of the fine-tuned Gemma 2 model trained on human annotations.
• Figure 5: Frequency of communicative feedback in synthetic dialogues generated using GPT-2 models, either applied without fine-tuning or after fine-tuning on corpora of spontaneous interactions or subtitles.