Language Evolution with Deep Learning

TL;DR

This chapter investigates how deep learning and reinforcement learning can model language emergence by casting referential and negotiation tasks as multi-agent learning problems. It outlines a framework for building neural agents with perception, generation, understanding, and action modules that solve communication games, and discusses optimization strategies including supervised and reinforcement learning, loss functions, and regularization. A Visual Discrimination Game case study demonstrates how neural agents develop emergent communication protocols, while the discussion addresses limitations of referential tasks and proposes more realistic scenarios, agents, and linguistically informed metrics. The chapter highlights the potential of deep learning to scale language evolution simulations and suggests reciprocal benefits for natural language processing and cognitive science.

Abstract

Computational modeling plays an essential role in the study of language emergence. It aims to simulate the conditions and learning processes that could trigger the emergence of a structured language within a simulated controlled environment. Several methods have been used to investigate the origin of our language, including agent-based systems, Bayesian agents, genetic algorithms, and rule-based systems. This chapter explores another class of computational models that have recently revolutionized the field of machine learning: deep learning models. The chapter introduces the basic concepts of deep and reinforcement learning methods and summarizes their helpfulness for simulating language emergence. It also discusses the key findings, limitations, and recent attempts to build realistic simulations. This chapter targets linguists and cognitive scientists seeking an introduction to deep learning as a tool to investigate language evolution.

Figures (14)

• Figure 1: Iterative learning process in a machine learning problem. Step 1 (simulation): The computer program $f$ performs an experience $E$ of the task $T$. Step 2 (measure): The task's success is measured through a performance measure $P$. Step 3 (improvement): Based on its performance, the computer program $f$ update to improve its future performance, i.e. learns. Communication games can be framed as a machine learning problem by modeling agents as computer programs. The experience $E$ corresponds to an episode of the game, while a performance measure $P$ measures the game's success.
• Figure 2: Scheme of a two-player communication game. Step 1 (simulation): Agents play a round of the game: (1) both agents get observations from the environment, (2) The sender sends a message to the receiver, (3) The receiver uses the message and its observation to perform an action in the environment. Step 2 (measure): One reward signal per agent measures the game's success. Step 3 (improvement): Agents receive the reward signals and update their behavior toward better solving the game.
• Figure 3: Examples of Lewis and negotiation games. (a) Example of Lewis game object's attributes decomposition (shape, color, style). (b) In the Lewis reconstruction game, the sender observes an object composed of several independent attributes and describes it to the receiver. The receiver must then predict the initial object attributes. (c) In the Lewis discrimation game, the receiver must retrieve the object within a set of distractors. Such a setting does not require manually defining independent attributes, allowing the use of ambiguous real data inputs such as images. (a-c) Such Lewis games usually aim to explore the disentanglement skills of the sender toward producing a compositional language under different scenarios or learning pressures. (d) In negotiation games, agents value objects or attributes differently and get a set of initial objects. They then start dialoguing before executing a final trade. Such tasks involve diverse language interactions such as multi-turn communication, non-fully cooperative games, repeated games, or action binding.
• Figure 4: Attempts to go beyond Lewis and negotiation games by embodying agents into a 2D world. (a) In the following instruction tasks, the sender is aware of the extensive state of the world and must instruct the receiver on how to reach a predefined goal. Importantly, the receiver only has a partial view of its environment. Such tasks aim to explore how basic embodiment properties may shape communication. (b) In coordination games, the agent needs to communicate to execute joint tasks or improve coordination and success through communication. Such tasks ground language to actions. (c) In social dilemma games, agents are surrounded by multiple agents and must behave accordingly to survive. Such tasks explore multi-channel communication or behavioral communication through actions.
• Figure 5: General view of a communicative agent. A communicative agent is composed of four functional modules: a perception module that maps an observation to an internal representation; a generation module that maps an internal representations to a message; an understanding module that maps a message to an internal representation; an action module that maps internal representations to an action.