On Parallelism in Music and Language: A Perspective from Symbol Emergence Systems based on Probabilistic Generative Models

TL;DR

The paper investigates how symbol meanings can emerge from sensorimotor experience and social interaction, and how this process relates to parallelism between music and language. It surveys and extends probabilistic generative models for language learning and symbol emergence in robotics, introducing collective predictive coding and a decentralized Metropolis Hastings Naming Game. A central thesis is that meanings arise through social Bayesian inference, with emotion and interoception proposed as grounding mechanisms for musical meaning. The work provides a theoretical framework linking symbol emergence, predictive coding, and music-language parallels, with potential implications for cognitive robotics, music informatics, and AI semantics.

Abstract

Music and language are structurally similar. Such structural similarity is often explained by generative processes. This paper describes the recent development of probabilistic generative models (PGMs) for language learning and symbol emergence in robotics. Symbol emergence in robotics aims to develop a robot that can adapt to real-world environments and human linguistic communications and acquire language from sensorimotor information alone (i.e., in an unsupervised manner). This is regarded as a constructive approach to symbol emergence systems. To this end, a series of PGMs have been developed, including those for simultaneous phoneme and word discovery, lexical acquisition, object and spatial concept formation, and the emergence of a symbol system. By extending the models, a symbol emergence system comprising a multi-agent system in which a symbol system emerges is revealed to be modeled using PGMs. In this model, symbol emergence can be regarded as collective predictive coding. This paper expands on this idea by combining the theory that ''emotion is based on the predictive coding of interoceptive signals'' and ''symbol emergence systems,'' and describes the possible hypothesis of the emergence of meaning in music.

Figures (2)

• Figure 1: an overview of a symbol emergence system Taniguchi2016SERtaniguchi2018symbol
• Figure 2: (Left) Diagram of the process symbol emergence based on collective predictive coding. Each agent forms internal representations reflecting their perceptual state, and they use and form language through semiotic communications. The complete process is regarded as collective predictive coding taniguchi2022emergentHagiwara2019hagiwara2022multiagent. (Center) A PGM for symbol emergence systems corresponding to Inter-DM, Inter-MDM, and Inter-GMM+VAE Hagiwara2019hagiwara2022multiagenttaniguchi2022emergent. Metropolis Hastings naming game becomes a decentralized Bayesian inference of the shared $w$ and internal representations $z^A$ and $z^B$. Note that in this graphical model, head-to-head connection across $w$ is adopted KazumaFurukawa2022. (Right) Hypothetical diagram of the process of emergence of musical symbol systems. Instead of sensorimotor signals, emotional states are inferred through predictive coding of introspective signals seth2013interoceptivebarrett2015interoceptive. Such internal representations may become the basis of the emergence of a musical symbol system, i.e., the source of the socially constructed meaning of music.