The emergence of numerical representations in communicating artificial agents

TL;DR

This work investigates whether the pressure to communicate can spontaneously give rise to numerical representations in neural agents and whether emergent codes resemble human numerals. Two communication channels are studied: discrete tokens and continuous sketches, within a referential game lacking predefined numeric concepts. Agents achieve high in-distribution accuracy and precise symbol–meaning mappings, but the resulting codes are non-compositional and fail to generalise to unseen numerosities. The findings imply that while communication pressure suffices for precise numerosity transmission, additional pressures—such as those promoting compositionality and generalisation—are needed to boot robust numerical concepts.

Abstract

Human languages provide efficient systems for expressing numerosities, but whether the sheer pressure to communicate is enough for numerical representations to arise in artificial agents, and whether the emergent codes resemble human numerals at all, remains an open question. We study two neural network-based agents that must communicate numerosities in a referential game using either discrete tokens or continuous sketches, thus exploring both symbolic and iconic representations. Without any pre-defined numeric concepts, the agents achieve high in-distribution communication accuracy in both communication channels and converge on high-precision symbol-meaning mappings. However, the emergent code is non-compositional: the agents fail to derive systematic messages for unseen numerosities, typically reusing the symbol of the highest trained numerosity (discrete), or collapsing extrapolated values onto a single sketch (continuous). We conclude that the communication pressure alone suffices for precise transmission of learned numerosities, but additional pressures are needed to yield compositional codes and generalisation abilities.

Figures (10)

• Figure 1: (a) A referential game: the Receiver has to guess, based on a Sender's discrete message, the correct target out of 5 possible numerosities. Notice that although the target is also 2, it is not the same instance as seen by the Sender. (b) Examples of target numerosities $\{1,2,3,4,5\}$ and their 5-line corresponding sketches.
• Figure 2: (a) Discrete communication accuracy and (b) message entropy, and (c) continuous (i.e. sketch-based) communication accuracy for setups involving identical target pictures (same - blue line) and different pictures showing the same numerosity (diff - green line). Model trained and tested on numerosities 1 to 5. Results averaged over 3 seeds.
• Figure 3: Discrete Communication - Agent messaging behaviour when longer messages are penalised by regularising the game loss. Model trained on numerosities $\{1,2,3,4,5\}$, with a variable message length allowed up to 5 tokens, and $vocabulary=3$. (a) Accuracy (b) conditional entropy of the mapping, and (c) average message length for different regularisation coefficients.
• Figure 4: (a) Conditional entropy for communication using 1 and 5-line sketches with two target contexts (same or different between the 2 agents). (b) Communication accuracy on two altered numerosity datasets with an unequal number of training samples: increasing from class 1, having the fewest samples, to class 5, having the most, and vice versa. (c) Test sketches from training on the 3 different datasets: equal, increasing and decreasing number of class samples.
• Figure 5: (Left) Discrete extrapolation: model trained on $\{1,2,3,4,5\}$. Test communication accuracy is reported for each test set (train $+$ one out-of-distribution numerosity). The model uses the same message that was previously chosen for the highest pretrained numerosity (e.g. 5) for every new higher extrapolation. (Right) Interpolation results from two different training sets. (Bottom) Example of sketch extrapolations from training within $\{1-20\}$ to 25 highlighted in red.