MusicAOG: an Energy-Based Model for Learning and Sampling a Hierarchical Representation of Symbolic Music

TL;DR

MusicAOG presents an energy-based, hierarchical symbolic-music framework that unifies explicit and implicit musical information across cultures. By formulating a minimax entropy objective and employing Metropolis-Hastings sampling, it achieves interpretable learning and controllable generation within an attributed And-Or Graph representation. The approach demonstrates favorable subjective evaluations and cross-cultural representability, highlighting its potential for interpretable analysis, composition, and cross-notational musicology. This work advances symbolic music modeling by coupling a rich hierarchical grammar with principled probabilistic learning and controlled synthesis.

Abstract

In addressing the challenge of interpretability and generalizability of artificial music intelligence, this paper introduces a novel symbolic representation that amalgamates both explicit and implicit musical information across diverse traditions and granularities. Utilizing a hierarchical and-or graph representation, the model employs nodes and edges to encapsulate a broad spectrum of musical elements, including structures, textures, rhythms, and harmonies. This hierarchical approach expands the representability across various scales of music. This representation serves as the foundation for an energy-based model, uniquely tailored to learn musical concepts through a flexible algorithm framework relying on the minimax entropy principle. Utilizing an adapted Metropolis-Hastings sampling technique, the model enables fine-grained control over music generation. A comprehensive empirical evaluation, contrasting this novel approach with existing methodologies, manifests considerable advancements in interpretability and controllability. This study marks a substantial contribution to the fields of music analysis, composition, and computational musicology.

Figures (5)

• Figure 1: An example music parse graph of Robert Schumann's Kinderszenen, Op.15, No.1: Von fremden Ländern und Menschen. The section, period, phrase, radical, note and metrical node are nodes on the parse graph, while the succession, variation, diad, and synchronization are relations. They are arranged hirarchically and horizontally throu production rules and relations. The metrial tree are implicit tree where each metrical nodes are linked to nodes in event trees with synthronization relations.
• Figure 2:
• Figure 3:
• Figure 4:
• Figure 6: Gonchepu representation (top) and its representation in MusicAOG (bottom)