Partner Modelling Emerges in Recurrent Agents (But Only When It Matters)

TL;DR

The study asks whether partner modelling can spontaneously emerge in model-free recurrent agents engaged in cooperative tasks. By training GRU-based ego agents with diverse partners in Overcooked-AI, the authors show that internal representations of partners' abilities arise without explicit modelling incentives, and that these representations support rapid adaptation to unseen and online-changing partners. The emergence hinges on environmental social pressure, notably when agents can influence partner task allocation, and even blind agents can develop partner models. Collectively, the results demonstrate that social predictive representations can arise from cooperative pressures and general learning mechanisms, highlighting a path toward more flexible, human-like collaboration in AI agents.

Abstract

Humans are remarkably adept at collaboration, able to infer the strengths and weaknesses of new partners in order to work successfully towards shared goals. To build AI systems with this capability, we must first understand its building blocks: does such flexibility require explicit, dedicated mechanisms for modelling others -- or can it emerge spontaneously from the pressures of open-ended cooperative interaction? To investigate this question, we train simple model-free RNN agents to collaborate with a population of diverse partners. Using the `Overcooked-AI' environment, we collect data from thousands of collaborative teams, and analyse agents' internal hidden states. Despite a lack of additional architectural features, inductive biases, or auxiliary objectives, the agents nevertheless develop structured internal representations of their partners' task abilities, enabling rapid adaptation and generalisation to novel collaborators. We investigated these internal models through probing techniques, and large-scale behavioural analysis. Notably, we find that structured partner modelling emerges when agents can influence partner behaviour by controlling task allocation. Our results show that partner modelling can arise spontaneously in model-free agents -- but only under environmental conditions that impose the right kind of social pressure.

Figures (8)

• Figure 1: An illustration of our task setting, based on the 'cramped room' layout of Overcooked-AI. The ego agent (green) has a learned internal representation of how competent (fast) their partner (blue) is at each of the two subtasks; it uses this representation to determine which subtask the partner should work on.
• Figure 2: Comparison between different ego agent policies in the overcooked environment. Each policy was evaluated in five different layouts against 8 different combinations of partner speed parameters, over 10 seeds. (A) average episode reward, per-layout and overall (B) throughput (rate of soup production), per-layout and overall, with shaded areas giving bootstrapped 95% confidence intervals (C) correlation between how much faster the partner agent was at task 1 vs task 2 and the proportion of time the partner spent performing task 1 (over all layouts). Shaded areas show 95% confidence intervals over the slope; a higher correlation indicates more efficient task allocation.
• Figure 3: A comparison of the hidden states of RNN ego agents trained under different conditions. (A) UMAP embeddings of RNN hidden states averaged over the final 50 timesteps of each episode, coloured by the partner's difference in task speeds (speed 1 $-$ speed 2), for five different layouts of the Overcooked environment. (B) Mean test accuracy of linear probes trained to recover partner speeds from sets of RNN hidden states accumulated up to different timesteps (with shaded areas giving bootstraped 95% confidence intervals).
• Figure 4: A demonstration of online adaptation. (A) Average throughput (rate of soup delivery) during episodes where the partner is switched halfway through from being faster at task 1 to faster at task 2. (B) From the same episodes, the average proportion of time spent by the partner performing each subtask before and after the switch. (C) UMAP embeddings of the average pre-switch and post-switch RNN hidden states from each episode. Also shown are the distributions (approximated via KDE) for embeddings of hidden states from non-switch baseline episodes with partners matching the pre- and post-switch speeds respectively.
• Figure 5: (A) An example CoinGame environment layout at episode start. The two coloured circles are coins, the red square is a teammate currently in 'collect red coins' mode and the black square represents the ego agent. (B) A comparison of the evaluation performance of different ego agents trained in the CoinGame environment, showing the mean total number of coins collected by ego agent and teammate (left) and the mean proportion of time spent by the teammate pursuing the 'correct' coin colour (i.e. the one matched to their highest skill level). Error bars give bootstrapped 95% confidence intervals over 5 random training seeds.