Searching for Structure: Investigating Emergent Communication with Large Language Models

TL;DR

This study tests whether artificial languages evolve structure when optimized for large language models in a referential game. Using instruction tuned Llama 3 70B with in context prompts, two agents learn and gradually shape vocabularies, yielding increased structure and improved generalisation alongside nonhumanlike degeneracy. The results align with human findings on learnability and structure but reveal biases unique to LLM learners that can amplify degeneracy under iterated learning. Overall, the work demonstrates the feasibility of using LLMs to simulate language evolution and highlights directions for human–machine collaborative experiments to further probe structure formation.

Abstract

Human languages have evolved to be structured through repeated language learning and use. These processes introduce biases that operate during language acquisition and shape linguistic systems toward communicative efficiency. In this paper, we investigate whether the same happens if artificial languages are optimised for implicit biases of Large Language Models (LLMs). To this end, we simulate a classical referential game in which LLMs learn and use artificial languages. Our results show that initially unstructured holistic languages are indeed shaped to have some structural properties that allow two LLM agents to communicate successfully. Similar to observations in human experiments, generational transmission increases the learnability of languages, but can at the same time result in non-humanlike degenerate vocabularies. Taken together, this work extends experimental findings, shows that LLMs can be used as tools in simulations of language evolution, and opens possibilities for future human-machine experiments in this field.

Figures (8)

• Figure 1: A graphical representation of the experimental blocks. The agents first go through a guessing block before labelling each of the 15 training stimuli in the labelling block. The communication block is done for 4 rounds each consisting of 30 tasks $T$, where the agents alternate speaker-listener roles to be a speaker and listener for each stimulus once. Finally, the agents label 27 (15 original and 12 novel) stimuli in the testing block.
• Figure 2: A vocabulary snippet used in a completion prompt. Full prompts are visible in Appendix \ref{['sec:prompts']}
• Figure 3: Communication clearly increases the structure of the vocabularies, as seen by the increasing TopSim scores and decreasing Ngram diversity.
• Figure 4: Languages that have evolved to be more structured allow for better generalisation to unseen test stimuli. Colours refer to individual simulations.
• Figure 5: The normalised Levenshtein distance between the ground truth and the produced signal in the learning block. Solid lines indicate chains, dashed blue line indicates the average Levenshtein distance across chains.