Data Melodification FM: Where Musical Rhetoric Meets Sonification
Ke Er Amy Zhang, David Grellscheid, Laura Garrison
TL;DR
Data melodification proposes mapping visualization idioms and immutable data characteristics to classical music rhetoric to create aesthetically pleasing, informative audio representations of data. Unlike traditional sonification that often emphasizes numerical parameters, this approach maps data trends to pitch, density to rhythm and reverb, and variance to pitch range, while aligning bar, line, pie, and scatter patterns with melodic constructs such as chords, arpeggios, and cadences. The authors provide a concise music theory primer and report both live-coding and tactile studio explorations, including a tracklist demonstrating varied melodic mappings. They argue for a human-in-the-loop workflow that balances mathematical structure and expressive aesthetics, acknowledging precision limitations but highlighting potential for accessible data storytelling and outreach.
Abstract
We propose a design space for data melodification, where standard visualization idioms and fundamental data characteristics map to rhetorical devices of music for a more affective experience of data. Traditional data sonification transforms data into sound by mapping it to different parameters such as pitch, volume, and duration. Often and regrettably, this mapping leaves behind melody, harmony, rhythm and other musical devices that compose the centuries-long persuasive and expressive power of music. What results is the occasional, unintentional sense of tinnitus and horror film-like impending doom caused by a disconnect between the semantics of data and sound. Through this work we ask, can the aestheticization of sonification through (classical) music theory make data simultaneously accessible, meaningful, and pleasing to one's ears?