Speech-based Mark for Data Sonification

TL;DR

The paper tackles data accessibility for visually impaired readers by introducing SpeechTone, a speech-based mark for data sonification that extends the Erie declarative grammar. SpeechTone encodes data directly into speech attributes—pitch, speed, voice, and spoken text—through four channels, enabling expressive, auditory data representations. Three demonstrations using the Cars dataset illustrate mappings from origin, year, and fuel efficiency to speech properties, showcasing how pitch, rate, and voice can convey quantitative and nominal attributes. While promising for efficiency and memorability, the work calls for empirical validation with screen-reader users and further development of interactive and multimodal capabilities. Overall, SpeechTone provides a concrete, extensible approach to making data sonification more accessible and expressive.

Abstract

Sonification serves as a powerful tool for data accessibility, especially for people with vision loss. Among various modalities, speech is a familiar means of communication similar to the role of text in visualization. However, speech-based sonification is underexplored. We introduce SpeechTone, a novel speech-based mark for data sonification and extension to the existing Erie declarative grammar for sonification. It encodes data into speech attributes such as pitch, speed, voice and speech content. We demonstrate the efficacy of SpeechTone through three examples.