Tonal Cognition in Sonification: Exploring the Needs of Practitioners in Sonic Interaction Design

TL;DR

A study of designers is reported on to understand the sound creation practices of industry experts and explore how infusing tonal music principles into a sound design tool can better support their craft and enhance the sonic experiences they create.

Abstract

Research into tonal music examines the structural relationships among sounds and how they align with our auditory perception. The exploration of integrating tonal cognition into sonic interaction design, particularly for practitioners lacking extensive musical knowledge, and developing an accessible software tool, remains limited. We report on a study of designers to understand the sound creation practices of industry experts and explore how infusing tonal music principles into a sound design tool can better support their craft and enhance the sonic experiences they create. Our study collected qualitative data through semi-structured individual and focus group interviews with six participants. We developed a low-fidelity prototype sound design tool that involves practical methods of functional harmony and interaction design discussed in focus groups. We identified four themes through reflexive thematic analysis: decision-making, domain knowledge and terminology, collaboration, and contexts in sound creation. Finally, we discussed design considerations for an accessible sonic interaction design tool that aligns auditory experience more closely with tonal cognition.

Figures (8)

• Figure 1: Background overview including fundamental concepts, corresponding methods, common ground, and following benefits; musical expectancy and movement sonification are the main points from Newbold et al. newbold2020movement, and the bigger frame represents the potential software development for designers reflecting this sonification strategy.
• Figure 2: Method overview illustrating conceptual relationships, research scope of each method, and procedure for focus groups. The black frame represents overall sound design, while the gray frame specifically denotes sound design with tonal cognition. The two yellow boxes illustrate what each method encompasses: individual interviews cover both general sound design and sound design with tonal cognition, while focus groups concentrated exclusively on aspects of sound design with tonal cognition using a low-fidelity prototype framed by a blue box. The green boxes outline the procedure of the focus groups.
• Figure 3: Prototype design logic with user scenario formation, cognitive tonal functions arrangement to target tasks, tonal variable tuning, and file extraction within three main phases.
• Figure 4: Focus group description; a: interview environment with individual laptops and recording device, b: individual examination of prototype in Figma via Microsoft Forms, c: redesigning activity with paper-based prototype and pens.
• Figure 5: Four discovered reflexive themes comprising individual codes classified by thematic analysis; the colors indicate the source of each code, such as which participant or focus group it originated from, and when a code is mentioned by more than one participant or focus group, two colors are used with additional highlighting.