Image, Word and Thought: A More Challenging Language Task for the Iterated Learning Model

TL;DR

This paper extends the iterated learning model (ILM) by adopting a semi-supervised autoencoder framework to handle a complex, image-based meaning space built from seven-segment glyphs. It demonstrates that an outer and inner autoencoder can jointly learn perceptual structure and compositional language, enabling expressive, compositional, and stable languages to emerge and be transmitted under a learning bottleneck. The work provides evidence for a computational account of language of thought and suggests that autoencoder-driven learning is crucial for transmitting structured languages in richer meaning spaces. Together, these results broaden ILM applicability and offer a platform to investigate how cognitive representations interact with language evolution.

Abstract

The iterated learning model simulates the transmission of language from generation to generation in order to explore how the constraints imposed by language transmission facilitate the emergence of language structure. Despite each modelled language learner starting from a blank slate, the presence of a bottleneck limiting the number of utterances to which the learner is exposed can lead to the emergence of language that lacks ambiguity, is governed by grammatical rules, and is consistent over successive generations, that is, one that is expressive, compositional and stable. The recent introduction of a more computationally tractable and ecologically valid semi supervised iterated learning model, combining supervised and unsupervised learning within an autoencoder architecture, has enabled exploration of language transmission dynamics for much larger meaning-signal spaces. Here, for the first time, the model has been successfully applied to a language learning task involving the communication of much more complex meanings: seven-segment display images. Agents in this model are able to learn and transmit a language that is expressive: distinct codes are employed for all 128 glyphs; compositional: signal components consistently map to meaning components, and stable: the language does not change from generation to generation.

Figures (7)

• Figure 1: Example base glyphs. Each base glyph is a $28\times 28$ image based on the classic seven-segment LED display. All 128 glyphs are employed, many of which do not correspond to digits or letters. The three glyph types presented here correspond to digits 1, 5, and 8. Images are presented as flattened 784-vectors with components between zero and one. The colours used here are for the purpose of illustration, selected to resemble a digital clock display.
• Figure 2: Example images. For each base glyph, 100 variant images were produced using small random rotations, translations and changes in intensity. Example images from each of the three noise conditions described in the text are shown here: A: noise 1, B: noise 2 and C: noise 3.
• Figure 3: An expressive, compositional and stable language evolves. Thin lines show expressivity, compositionality and stability for ten individual 100-generation instantiations with $n_l=7$. Thick lines show the average values; all three measures are defined so that they range from zero to one.
• Figure 4: A compositional and expressive language is easier to learn. Loss has been plotted against epoch during pupil training. Four different parts of a pupil's network for $n_l=7$ are trained for $g=30$ generations and the loss for each of these is plotted here; different generations distinguished by color and warmer colors corresponding to later generations. The loss for the first epoch of the first pupil is used to normalize the plots and ten instantiations have been averaged.
• Figure 5: Reconstructed images for the 5 glyph. The top row presents reconstructions after six generations, the bottom row, after 30. Three different types of reconstruction are shown: the first column, outer, shows reconstruction by the outer autoencoder, $O$; the second, autoencoder, shows reconstruction by the total autoencoder, $A$. The third column, decoder is similar, it is $D\circ E$ so it differs from the autoencoder by including the discretization step. After six generations the network is already making compositional mistakes, inclusion or omission of entire segments, but the different types of autoencoding are not fully aligned to each other. By generation 30 the reconstructions are nearly identical. This example used $n_l=10$.