Factorised Active Inference for Strategic Multi-Agent Interactions

TL;DR

This work integrates Active Inference with game theory by factorising the generative model so that an agent maintains individual beliefs about other agents’ internal states and uses them for strategic planning in a joint context. The approach is applied to iterated general-sum games with two and three players, exploring non-stationary payoffs and equilibria via variational free energy (VFE) and expected free energy (EFE) dynamics. Key findings show that ensemble-level EFE characterises basins of attraction for multiple Nash equilibria and that cooperation can emerge or be suppressed depending on transitions and trust dynamics, not always aligning with the aggregate minimisation of EFE. The results provide a framework for analyzing how intelligent collectives learn, adapt, and optimize actions in dynamic social environments, with implications for designing interventions that leverage trust rather than punishment to achieve favorable outcomes.

Abstract

Understanding how individual agents make strategic decisions within collectives is important for advancing fields as diverse as economics, neuroscience, and multi-agent systems. Two complementary approaches can be integrated to this end. The Active Inference framework (AIF) describes how agents employ a generative model to adapt their beliefs about and behaviour within their environment. Game theory formalises strategic interactions between agents with potentially competing objectives. To bridge the gap between the two, we propose a factorisation of the generative model whereby each agent maintains explicit, individual-level beliefs about the internal states of other agents, and uses them for strategic planning in a joint context. We apply our model to iterated general-sum games with two and three players, and study the ensemble effects of game transitions, where the agents' preferences (game payoffs) change over time. This non-stationarity, beyond that caused by reciprocal adaptation, reflects a more naturalistic environment in which agents need to adapt to changing social contexts. Finally, we present a dynamical analysis of key AIF quantities: the variational free energy (VFE) and the expected free energy (EFE) from numerical simulation data. The ensemble-level EFE allows us to characterise the basins of attraction of games with multiple Nash Equilibria under different conditions, and we find that it is not necessarily minimised at the aggregate level. By integrating AIF and game theory, we can gain deeper insights into how intelligent collectives emerge, learn, and optimise their actions in dynamic environments, both cooperative and non-cooperative.

Figures (4)

• Figure 1: The perception-action loop; ego's 'internal world', $\psi_i$. The agent observes the actions of all agents, from which she updates her beliefs $q(s)$ to minimise VFE. These beliefs are used to plan her next action by minimising EFE.
• Figure 2: Dynamics of a game transition with two agents ($\beta_1=15$). 500 steps of Ch followed by 500 steps of SH with a 10-step transition. In the EFE plots, blue represents 'cooperate', and pink represents 'defect'. In the policy heatmap plots, a lighter colour indicates a higher probability.
• Figure 3: Stylised bounds on the dynamics of the VFE.
• Figure 4: The ensemble-level expected EFE, $\mathfrak{G}$, highlights (the relative size of the basin of attraction of) the equilibria of a game ($\beta_1 = 30$). The bottom-right plot shows the kernel density estimate (Gaussian kernel, 0.08 bandwidth) of the PDF of final values under each condition.