Exploring the Needs of Practising Musicians in Co-Creative AI Through Co-Design

TL;DR

This study addresses how co-creative AI for music can be aligned with practising musicians' needs. It employs a participatory co-design approach with 13 musicians across two workshops and a two-week ecological evaluation to develop a MusicBERT/MidiFormers based music variation tool. Key findings show musicians prefer AI as a controllable tool rather than a full collaborator, stress ownership of the creative process, and highlight the importance of framing, diverse musical backgrounds, and DAW integration in tool design. The work demonstrates a practitioner-centered method for shaping co-creative musical AI early in development with practical guidance for future tool design and deployment.

Abstract

Recent advances in generative AI music have resulted in new technologies that are being framed as co-creative tools for musicians with early work demonstrating their potential to add to music practice. While the field has seen many valuable contributions, work that involves practising musicians in the design and development of these tools is limited, with the majority of work including them only once a tool has been developed. In this paper, we present a case study that explores the needs of practising musicians through the co-design of a musical variation system, highlighting the importance of involving a diverse range of musicians throughout the design process and uncovering various design insights. This was achieved through two workshops and a two week ecological evaluation, where musicians from different musical backgrounds offered valuable insights not only on a musical system's design but also on how a musical AI could be integrated into their musical practices.

Figures (4)

• Figure 1: Study workflow: Workshop 1 was used as the initial investigation into how a co-creative AI can be situated in a musical practice and informed the design of workshop 2. Following insights from the previous investigation, Workshop 2 was focused on understanding the use of a variation system, and with workshop 1, was used to continue guiding the system development. This system was then used in an ecological evaluation to better understand participants specific needs from the system.
• Figure 2: Figure a: Musician being briefed on the musical system and preparing to compose with it. Figure b: Example of prompt card used by musicians when composing. Figure c: Picture from the workshop discussions with a participant placing ideas on the board.
• Figure 3: Figure a: Workshop discussion on music variation tools. Figure b: Participant working on a composition with the system. Figure c: Participants composing music in a studio.
• Figure 4: Masking functionality performed by the system. In this diagram, the green notes are unmasked and will remain unchanged by the system. The dark notes are masked and will have one or more of their 8 attributes altered by the AI. The 8 attributes (Bar, Instrument, Position, Pitch, Duration, Velocity, Time Signature and Tempo) are listed to the right.