Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Can Authorship Attribution Models Distinguish Speakers in Speech Transcripts?

Cristina Aggazzotti, Nicholas Andrews, Elizabeth Allyn Smith

TL;DR

This paper investigates whether textual authorship attribution models can identify speakers from transcribed speech. It introduces a speaker-attribution benchmark on the Fisher English transcripts, evaluating multiple neural and non-neural baselines, including LUAR, CISR, SBERT, PANgrams, TF-IDF, and AdHominem, across encodings and topic-control settings. The study finds that text-based models transfer to speech transcripts in the base condition but performance collapses under topic control, with fine-tuning and domain-specific pretraining mitigating this decline. It also shows that transcription style matters, that performance plateaus after roughly $75$ utterances, and that pre-training on speech transcripts yields the strongest gains, suggesting potential practical use in forensic and archival contexts while underscoring limitations and ethical concerns.

Abstract

Authorship verification is the task of determining if two distinct writing samples share the same author and is typically concerned with the attribution of written text. In this paper, we explore the attribution of transcribed speech, which poses novel challenges. The main challenge is that many stylistic features, such as punctuation and capitalization, are not informative in this setting. On the other hand, transcribed speech exhibits other patterns, such as filler words and backchannels (e.g., 'um', 'uh-huh'), which may be characteristic of different speakers. We propose a new benchmark for speaker attribution focused on human-transcribed conversational speech transcripts. To limit spurious associations of speakers with topic, we employ both conversation prompts and speakers participating in the same conversation to construct verification trials of varying difficulties. We establish the state of the art on this new benchmark by comparing a suite of neural and non-neural baselines, finding that although written text attribution models achieve surprisingly good performance in certain settings, they perform markedly worse as conversational topic is increasingly controlled. We present analyses of the impact of transcription style on performance as well as the ability of fine-tuning on speech transcripts to improve performance.

Can Authorship Attribution Models Distinguish Speakers in Speech Transcripts?

TL;DR

This paper investigates whether textual authorship attribution models can identify speakers from transcribed speech. It introduces a speaker-attribution benchmark on the Fisher English transcripts, evaluating multiple neural and non-neural baselines, including LUAR, CISR, SBERT, PANgrams, TF-IDF, and AdHominem, across encodings and topic-control settings. The study finds that text-based models transfer to speech transcripts in the base condition but performance collapses under topic control, with fine-tuning and domain-specific pretraining mitigating this decline. It also shows that transcription style matters, that performance plateaus after roughly utterances, and that pre-training on speech transcripts yields the strongest gains, suggesting potential practical use in forensic and archival contexts while underscoring limitations and ethical concerns.

Abstract

Authorship verification is the task of determining if two distinct writing samples share the same author and is typically concerned with the attribution of written text. In this paper, we explore the attribution of transcribed speech, which poses novel challenges. The main challenge is that many stylistic features, such as punctuation and capitalization, are not informative in this setting. On the other hand, transcribed speech exhibits other patterns, such as filler words and backchannels (e.g., 'um', 'uh-huh'), which may be characteristic of different speakers. We propose a new benchmark for speaker attribution focused on human-transcribed conversational speech transcripts. To limit spurious associations of speakers with topic, we employ both conversation prompts and speakers participating in the same conversation to construct verification trials of varying difficulties. We establish the state of the art on this new benchmark by comparing a suite of neural and non-neural baselines, finding that although written text attribution models achieve surprisingly good performance in certain settings, they perform markedly worse as conversational topic is increasingly controlled. We present analyses of the impact of transcription style on performance as well as the ability of fine-tuning on speech transcripts to improve performance.
Paper Structure (23 sections, 3 figures, 10 tables)

This paper contains 23 sections, 3 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. A Speaker Attribution Benchmark
  4. Experiments
  5. Experiment 1: Baselines
  6. Out-of-the-box attribution models transfer to speech transcripts (without topic control).
  7. Experiment 2: Transcription style
  8. Superficial text features are not needed.
  9. Removing transcript annotations can improve performance.
  10. Experiment 3: Topic
  11. Out-of-the-box performance degrades with topic control.
  12. Experiment 4: Fine-tuning
  13. Fine-tuning helps if train-test distributions match.
  14. Experiment 5: Pre-training domain
  15. Pre-training on speech transcripts performs best.
  16. ...and 8 more sections

Figures (3)

  • Figure 1: Examples of the two Fisher transcript encodings, 'BBN' and 'LDC'.
  • Figure 2: Bootstrapped AUC test performance (y-axis) across out-of-the-box and fine-tuned models (columns) on the LDC encoding at the 3 levels of difficulty (rows) with the number of utterances per speaker varied (x-axis). Increasing the number of utterances improves performance for all models, with the best generally achieved by 135 utterances.
  • Figure 3: Bootstrapped AUC test performance (y-axis) across out-of-the-box and fine-tuned models (columns) on the BBN encoding at the 3 levels of difficulty (rows) with the number of utterances per speaker varied (x-axis). Increasing the number of utterances improves performance for all models, with the best generally achieved by 135 utterances.