Read the Room: Adapting a Robot's Voice to Ambient and Social Contexts

TL;DR

This paper addresses how to adapt a robot's voice to ambient and social contexts to enhance perceived intelligence and social awareness. It introduces a data-driven pipeline: collect ambiance-specific voice data via Zoom, cluster human vocal styles to derive a small set of robot voice styles, and validate these styles with Pepper in recreated ambiances. The study shows that voice style choice significantly affects perceived social appropriateness, ambiance awareness, comfort, human-likeness, and competence, with context-dependent preferences (e.g., Neutral best in fine dining, Joyful in high-arousal settings). A Lombard-inspired, high-pitch voice offers some advantages but does not universally outperform the Joyful voice; overall, the three-style mapping provides an interpretable, effective approach for context-aware robot voice design and paves the way for future real-time adaptation and cross-robot applicability.

Abstract

How should a robot speak in a formal, quiet and dark, or a bright, lively and noisy environment? By designing robots to speak in a more social and ambient-appropriate manner we can improve perceived awareness and intelligence for these agents. We describe a process and results toward selecting robot voice styles for perceived social appropriateness and ambiance awareness. Understanding how humans adapt their voices in different acoustic settings can be challenging due to difficulties in voice capture in the wild. Our approach includes 3 steps: (a) Collecting and validating voice data interactions in virtual Zoom ambiances, (b) Exploration and clustering human vocal utterances to identify primary voice styles, and (c) Testing robot voice styles in recreated ambiances using projections, lighting and sound. We focus on food service scenarios as a proof-of-concept setting. We provide results using the Pepper robot's voice with different styles, towards robots that speak in a contextually appropriate and adaptive manner. Our results with N=120 participants provide evidence that the choice of voice style in different ambiances impacted a robot's perceived intelligence in several factors including: social appropriateness, comfort, awareness, human-likeness and competency.

Figures (10)

• Figure 1: Robots may be deployed in varied ambiances, from cozy formal dining to loud nightclubs. How should their voices change?
• Figure 2: Zoom virtual backgrounds and ambient sounds through headphones were used for data collection.
• Figure 3: Audio features across speakers as a difference from the baseline.
• Figure 4: Which voice styles are associated with which ambiance? We visualize the proportion of each ambiance's utterances represented by each voice style.
• Figure 5: What do the voice styles sound like? Feature analysis of the three voice styles derived from human voice cluster centers.