Music-triggered fashion design: from songs to the metaverse

TL;DR

The paper investigates how virtual realities can extend musical aesthetics into interactive fashion design, enabling real-time, music-driven clothing changes in the metaverse. It proposes a music-triggered fashion-design pipeline that derives color palettes from song-associated images via web scraping and $k=10$-means clustering, then uses parametric Rhino/Grasshopper tools to generate clothing patterns. It discusses the inherent ambiguity and biases in mapping audio features to visuals, and it reflects on social implications and authenticity concerns of virtual concerts. Code and a demonstration are provided to enable future research and practical exploration in artist-audience interaction in virtual spaces.

Abstract

The advent of increasingly-growing virtual realities poses unprecedented opportunities and challenges to different societies. Artistic collectives are not an exception, and we here aim to put special attention into musicians. Compositions, lyrics and even show-advertisements are constituents of a message that artists transmit about their reality. As such, artistic creations are ultimately linked to feelings and emotions, with aesthetics playing a crucial role when it comes to transmit artist's intentions. In this context, we here analyze how virtual realities can help to broaden the opportunities for musicians to bridge with their audiences, by devising a dynamical fashion-design recommendation system inspired by sound stimulus. We present our first steps towards re-defining musical experiences in the metaverse, opening up alternative opportunities for artists to connect both with real and virtual (\textit{e.g.} machine-learning agents operating in the metaverse) in potentially broader ways.

Figures (4)

• Figure 1: Music-triggered recommendation system. We show several designs obtained by our recommendation system, each associated to a different songs sample. Here, song features are extracted and further processed by our model, that imprints visual patterns into an avatar's metaverse dressing code. In this example, we show T-Shirt patterns of different songs based on: Moon Song by Phoebe Bridgers, 7 rings by Ariana Grande, Bad Guy by Billie Eilish and August by Taylor Swift.
• Figure 2: Recommendation system workflow with Rhinoceros generation of Super Freaky Girl. Grasshoppers interface is shown on the left, linking the colour palette with the song. On the right, the first image appearing on the google search Super Freaky Girl Nicki Minaj is shown, along with the generated colour-palette and the T-shirt example.
• Figure 3: Generated T-shirts from songs from the same album. Comparison between Need to Know by Doja Cat (left) and You Right by Doja Cat featuring The Weeknd (right), both from Doja Cat's third studio album Planet Her (centre).
• Figure 4: Clusters of songs based on colour palettes. Comparison between Butterflies by Kacey Musgraves (a) and Psychosocial by Slipknot (b).