Musical Agent Systems: MACAT and MACataRT
Keon Ju M. Lee, Philippe Pasquier
TL;DR
MACAT and MACataRT address the need for artist-centered AI co-creation in live music by leveraging small data and corpus-based synthesis. The approach combines MASOM and CataRT inspired architectures with a factor oracle temporal model and audio mosaicing to enable real-time and proactive improvisation. The work demonstrates practical viability through live performances and contest recognition, and emphasizes ethical advantages of transparent, personalized training data. It shows that adaptive, interpretable AI co-creators can broaden performance practice while reducing environmental impact and copyright concerns.
Abstract
Our research explores the development and application of musical agents, human-in-the-loop generative AI systems designed to support music performance and improvisation within co-creative spaces. We introduce MACAT and MACataRT, two distinct musical agent systems crafted to enhance interactive music-making between human musicians and AI. MACAT is optimized for agent-led performance, employing real-time synthesis and self-listening to shape its output autonomously, while MACataRT provides a flexible environment for collaborative improvisation through audio mosaicing and sequence-based learning. Both systems emphasize training on personalized, small datasets, fostering ethical and transparent AI engagement that respects artistic integrity. This research highlights how interactive, artist-centred generative AI can expand creative possibilities, empowering musicians to explore new forms of artistic expression in real-time, performance-driven and music improvisation contexts.