Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Creating an African American-Sounding TTS: Guidelines, Technical Challenges,and Surprising Evaluations

Claudio Pinhanez, Raul Fernandez, Marcelo Grave, Julio Nogima, Ron Hoory

TL;DR

This paper tackles the ethics and feasibility of constructing an African American TTS voice that is both professionally intelligible and recognizably African American. It outlines a voice-talent selection process, records a targeted conversational corpus, and implements a two-component TTS system (encoder–decoder acoustic model with a speaker-specific LPCNet vocoder) to generate AA voice prototypes. Through three large-scale studies, it reveals a striking dissociation: African American participants recognize the AA voice, whereas a generic U.S. listener base tends to misattribute it to White speakers, likely driven by stereotypes about Black speech. The work concludes that ethical representation challenges complicate achieving universal recognizability and suggests evaluating identity cues across modalities to preserve inclusivity without reinforcing harmful stereotypes.

Abstract

Representations of AI agents in user interfaces and robotics are predominantly White, not only in terms of facial and skin features, but also in the synthetic voices they use. In this paper we explore some unexpected challenges in the representation of race we found in the process of developing an U.S. English Text-to-Speech (TTS) system aimed to sound like an educated, professional, regional accent-free African American woman. The paper starts by presenting the results of focus groups with African American IT professionals where guidelines and challenges for the creation of a representative and appropriate TTS system were discussed and gathered, followed by a discussion about some of the technical difficulties faced by the TTS system developers. We then describe two studies with U.S. English speakers where the participants were not able to attribute the correct race to the African American TTS voice while overwhelmingly correctly recognizing the race of a White TTS system of similar quality. A focus group with African American IT workers not only confirmed the representativeness of the African American voice we built, but also suggested that the surprising recognition results may have been caused by the inability or the latent prejudice from non-African Americans to associate educated, non-vernacular, professionally-sounding voices to African American people.

Creating an African American-Sounding TTS: Guidelines, Technical Challenges,and Surprising Evaluations

TL;DR

This paper tackles the ethics and feasibility of constructing an African American TTS voice that is both professionally intelligible and recognizably African American. It outlines a voice-talent selection process, records a targeted conversational corpus, and implements a two-component TTS system (encoder–decoder acoustic model with a speaker-specific LPCNet vocoder) to generate AA voice prototypes. Through three large-scale studies, it reveals a striking dissociation: African American participants recognize the AA voice, whereas a generic U.S. listener base tends to misattribute it to White speakers, likely driven by stereotypes about Black speech. The work concludes that ethical representation challenges complicate achieving universal recognizability and suggests evaluating identity cues across modalities to preserve inclusivity without reinforcing harmful stereotypes.

Abstract

Representations of AI agents in user interfaces and robotics are predominantly White, not only in terms of facial and skin features, but also in the synthetic voices they use. In this paper we explore some unexpected challenges in the representation of race we found in the process of developing an U.S. English Text-to-Speech (TTS) system aimed to sound like an educated, professional, regional accent-free African American woman. The paper starts by presenting the results of focus groups with African American IT professionals where guidelines and challenges for the creation of a representative and appropriate TTS system were discussed and gathered, followed by a discussion about some of the technical difficulties faced by the TTS system developers. We then describe two studies with U.S. English speakers where the participants were not able to attribute the correct race to the African American TTS voice while overwhelmingly correctly recognizing the race of a White TTS system of similar quality. A focus group with African American IT workers not only confirmed the representativeness of the African American voice we built, but also suggested that the surprising recognition results may have been caused by the inability or the latent prejudice from non-African Americans to associate educated, non-vernacular, professionally-sounding voices to African American people.
Paper Structure (32 sections, 7 figures, 5 tables)

This paper contains 32 sections, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Whiteness in AI and NLP
  4. Diversity in Synthesized Speech
  5. Impacts of Race and Cultural Background on User Conversational Experience
  6. Initial Study: Development Guidelines, Selection Criteria, and Voice Talent Ranking
  7. Auditions
  8. Focus Groups Methodology
  9. Results and Findings
  10. Discussion
  11. Creating a State-of-the-Art TTS System from Scripted Recordings
  12. Creation of a TTS Conversational Corpus: Recording Script and Recordings
  13. Overview of the TTS System
  14. Generating the Synthetic Prototypes
  15. Evaluating Whether the AA Voice Sounds African American
  16. ...and 17 more sections

Figures (7)

  • Figure 1: Text-to-Speech Architecture. An acoustic model (blue) predicts spectral sequences from input text, which are then sent to an independent neural vocoder (yellow) to generate final speech.
  • Figure 2: Screen interface of the main task performed by the participants of the Study 1.
  • Figure 3: Synthetic faces used in Study 2.
  • Figure 4: Screen interface of the main task performed by the participants of the Study 2.
  • Figure 5: Candidate images for AA, WH, and AS evaluated by the participants in the study; highlighted images were selected to be used in Study 2.
  • ...and 2 more figures