Agent-based imitation dynamics can yield efficiently compressed population-level vocabularies

Abstract

Natural languages have been argued to evolve under pressure to efficiently compress meanings into words by optimizing the Information Bottleneck (IB) complexity-accuracy tradeoff. However, the underlying social dynamics that could drive the optimization of a language's vocabulary towards efficiency remain largely unknown. In parallel, evolutionary game theory has been invoked to explain the emergence of language from rudimentary agent-level dynamics, but it has not yet been tested whether such an approach can lead to efficient compression in the IB sense. Here, we provide a unified model integrating evolutionary game theory with the IB framework and show how near-optimal compression can arise in a population through an independently motivated dynamic of imprecise strategy imitation in signaling games. We find that key parameters of the model -- namely, those that regulate precision in these games, as well as players' tendency to confuse similar states -- lead to constrained variation of the tradeoffs achieved by emergent vocabularies. Our results suggest that evolutionary game dynamics could potentially provide a mechanistic basis for the evolution of vocabularies with information-theoretically optimal and empirically attested properties.

Figures (6)

• Figure 1: A. The IB communication model of zaslavskyEfficientCompressionColor2018, see \ref{['sec:ib']} for details. B. Communication model unifying noisy sim-max games with the IB framework. We illustrate our model on a semantic domain similar to numerals, and depict states as cards with varying quantities of dots, and meanings as (conditional) belief distributions about these states. We denote the (symmetric) noisy perception and interpretation of states as $p$. C. The imprecise conditional imitation dynamic frankeVaguenessImpreciseImitation2018. See main text and Appendix \ref{['app:dynamics']} for details.
• Figure 2: Emergent efficient semantic systems. A: Converged systems on the information plane, colored by the pragmatic standard of precision $\gamma$ in the sim-max game. Circles denote systems evolved from our model, derived from frankeVaguenessImpreciseImitation2018, abbreviated as FC18. See https://www.dropbox.com/scl/fi/vatlo672rmdqzf49s8uf8/trajectory_on_bound.mp4?rlkey=ilnbzte18kigwdcifgt4hlrsy&st=wco4ozim&dl=0 visualizing the dynamics on the plane. Gray circles mark the result of permuting the systems evolved from the FC18 dynamics (see Appendix \ref{['app:baselines']}). Gray triangles correspond to emergent systems evolved from a dynamic proposed in nowakEvolutionLanguage1999, abbreviated as NK99, which we consider as another baseline. B: Game pragmatic standard of precision ($\gamma$, \ref{['eq:similarity']}) vs. efficiency loss ($\epsilon$, \ref{['eq:epsilon']}) of converged systems. Mean scores for systems emerging from our model are depicted in blue while mean scores for permuted counterparts are in gray. Shaded regions correspond to the $95\%$ confidence interval. The inset shows a zoomed-in view of the emergent systems' scores only.
• Figure 3: Sample of converged emergent systems and their optimal counterparts. A-C: 1-dimensional mode maps of converged system for $\gamma=0.001, 0.01, 10$ (\ref{['eq:similarity']}). In each plot, the $x$-axis denotes the meaning space $\mathcal{X}_o = \{0, \dots, 99\}$, the $y$-axis denotes probability $S(w|m_o)$, and lines are the modal word $w$ (out of $100$ possible words) used for each meaning $m_o = x_o$. The color of each line corresponds to the average meaning that the word is used to communicate, with black corresponding to $x_1$ and bright orange corresponding to $x_{100}$. D-F: Mode maps for the optimal counterparts of the emergent systems in D-F, fitted by efficiency loss $\epsilon$ (\ref{['eq:epsilon']}). See https://www.dropbox.com/scl/fi/igdumose4bi0qx13vcfzj/system_movie.mov?rlkey=s3tx4mjylp86b8nemlp13zcni&st=x03po5qp&dl=0 for the evolution of the system in A.
• Figure 4: Complexity (A), accuracy (B), population fitness (C), and efficiency loss (D), relative to game pragmatic standard of precision ($\gamma$) across discrete time step of simulated evolution. Lines in each plot show mean (with 95% CIs) across eight random seeds. Color corresponds to $\gamma$ (\ref{['eq:similarity']}).
• Figure 5: The imprecise conditional imitation dynamic frankeVaguenessImpreciseImitation2018. The frequency of word $w$ being used to communicate state $x^{\mathrm{im}}_o$ at the next time step grows in proportion to (i) the expected frequency of $w$ given that $x^{\mathrm{im}}_o$ was observed by some (randomly sampled) imitating Sender, times (ii) the expected utility of this word relative to the population of Receivers. Likewise, the frequency of interpreting $w$ as meaning $\hat{x}^{\mathrm{im}}_o$ grows in proportion to (i) the probability that $\hat{x}^{\mathrm{im}}_o$ is observed by a randomly sampled agent imitating a randomly sampled Receiver that actually realized $\hat{x}_a$ after receiving word $w$, times (ii) the expected utility of the interpretation $\hat{x}^{\mathrm{im}}_o$ given $w$. See \ref{['eq:eu_sender', 'eq:imitation_prob_sender', 'eq:eu_receiver', 'eq:imitation_prob_receiver', 'eq:sender_update', 'eq:receiver_update']} for the details of these conditional expectations.