Purrfect Pitch: Exploring Musical Interval Learning through Multisensory Interfaces

TL;DR

This work tackles the challenge of musical ear training by integrating a back-worn vibrotactile vest with a web-based learning interface to train interval identification. Using a between-subjects study with eighteen novices, the authors compare audio-only and audio-plus-haptic conditions, finding a roughly 20 percentage-point accuracy boost and faster responses when haptics are present. The system demonstrates that multisensory cues can enhance perceptual learning, reduce frustration, and increase engagement, with effects persisting in training even after removing haptic feedback. The design is open-source and scalable, suggesting broad potential for multisensory approaches to perceptual skills beyond musical intervals.

Abstract

We introduce Purrfect Pitch, a system consisting of a wearable haptic device and a custom-designed learning interface for musical ear training. We focus on the ability to identify musical intervals (sequences of two musical notes), which is a perceptually ambiguous task that usually requires strenuous rote training. With our system, the user would hear a sequence of two tones while simultaneously receiving two corresponding vibrotactile stimuli on the back. Providing haptic feedback along the back makes the auditory distance between the two tones more salient, and the back-worn design is comfortable and unobtrusive. During training, the user receives multi-sensory feedback from our system and inputs their guessed interval value on our web-based learning interface. They see a green (otherwise red) screen for a correct guess with the correct interval value. Our study with 18 participants shows that our system enables novice learners to identify intervals more accurately and consistently than those who only received audio feedback, even after the haptic feedback is removed. We also share further insights on how to design a multisensory learning system.

Figures (11)

• Figure 1: The user experience procedure of a single trial. The user hears (and feel) two musical tones in sequence with a small gap in between. Then they type a number representing their guessed interval and receive on-screen feedback about their guess.
• Figure 2: Overview of the haptic device. It is a vest like wearable with eight vibrotactile modules evenly distributed along the spine. The haptic device can be controlled by our web-based learning interface via Bluetooth.
• Figure 3: The web-based learning interface that plays the musical notes and connects to the haptic device. The color green represents a correct answer, with a number displayed representing the correct interval number.
• Figure 4: Experimental protocol. The top lines represent the control condition and the bottom lines represent the experimental condition. The dashed lines represent when haptic feedback is absent and a solid line represents the presence of haptic feedback. The numbers in brackets represent the number of trials.
• Figure 5: Box plots of the perceived distance between two vibrotactile motors on the back. The cross mark indicates the mean value. Red dots and the red dashed line indicate the ground truth.