Constructive Approach to Bidirectional Influence between Qualia Structure and Language Emergence

TL;DR

It is suggested that language emergence serves not only as a mechanism for creating a communication tool but also as a mechanism for allowing people to realize shared understanding of qualitative experiences as a mechanism for allowing people to realize shared understanding of qualitative experiences.

Abstract

This perspective paper explores the bidirectional influence between language emergence and the relational structure of subjective experiences, termed qualia structure, and lays out a constructive approach to the intricate dependency between the two. We hypothesize that the emergence of languages with distributional semantics (e.g., syntactic-semantic structures) is linked to the coordination of internal representations shaped by experience, potentially facilitating more structured language through reciprocal influence. This hypothesized mutual dependency connects to recent advancements in AI and symbol emergence robotics, and is explored within this paper through theoretical frameworks such as the collective predictive coding. Computational studies show that neural network-based language models form systematically structured internal representations, and multimodal language models can share representations between language and perceptual information. This perspective suggests that language emergence serves not only as a mechanism creating a communication tool but also as a mechanism for allowing people to realize shared understanding of qualitative experiences. The paper discusses the implications of this bidirectional influence in the context of consciousness studies, linguistics, and cognitive science, and outlines future constructive research directions to further explore this dynamic relationship between language emergence and qualia structure.

Figures (3)

• Figure 1: (a) Emergence of language from internal representations. (b) Effect of language on internal representations. (c) Mutual influence between language and internal representations. Note that the existence of multiple agents (i.e., multiple brains) is crucial for the bottom-up organization of language, although we describe a single brain for each panel to avoid complicating the figure.Schematic diagrams illustrating bidirectional influence between language and internal representational structure. (a) Upward organization: Internal representational structure, formed through perceptual experience of the environment, influences language emergence. (b) Downward constraint: Language structure affects the structure of internal representations. (c) Bidirectional influence: Mutual interaction combining upward organization and downward constraint. Note: Each brain depicted internally forms and possesses a representational structure (illustrated schematically). The existence of multiple agents (i.e., multiple brains) is crucial for the actual bottom-up organization of language, although we describe a single brain for each panel to avoid complicating the figure.
• Figure 2: Shematic diagram of collective predictive coding.
• Figure 4: Structural influence in qualia and language: (A) Language-perception structural influence within an individual: The bidirectional arrow represents the mutual influence between the two qualia structures induced by language and perception. (B) Inter-personal structural influence: The diagram shows how language acts as a medium for aligning internal representations (qualia structures) between individuals A and B having a shared language system as a prior. The language system is updated through semiotic communicationcommunication using language over time.Bidirectional influence between language, perception, and internal representations. This figure illustrates how the structure of an individual's internal representations ($z$) is shaped by both bottom-up processing of perceptual observations ($o$) from the environment and top-down constraints from language (illustrated conceptually on the left and right sides). Simultaneously, the language emergence process shown in the center depicts how internal representations ($z^A, z^B$) formed by multiple agents (A, B) observing a shared environment can inform the emergence of a shared language system ($w$) through communicative interaction (e.g., language games). This emergent language system ($w$) then acts as a prior, imposing downward constraints that influence individual representations ($z^A, z^B$) and can promote their alignment across agents.