Realising Synthetic Active Inference Agents, Part II: Variational Message Updates

TL;DR

This work operationalises the Free Energy Principle for synthetic Active Inference by recasting perception, learning, and control as variational message passing on Constrained Forney-style Factor Graphs (CFFG). It derives general GFE-based message updates for a pair of facing nodes (goal prior and observation model) and provides concrete updates for a discrete-variable goal–observation submodel, including a T-maze demonstration. The approach yields epistemic behavior, allows learning of goal statistics, and supports multi-agent extensions such as bargaining, while acknowledging convergence caveats and the need for stochastic evaluation (importance sampling) in some updates. Overall, the paper offers a scalable, reusable set of message updates that move synthetic AIF closer to industrial applicability by unifying constraints and messages under a single GM and schedule.

Abstract

The Free Energy Principle (FEP) describes (biological) agents as minimising a variational Free Energy (FE) with respect to a generative model of their environment. Active Inference (AIF) is a corollary of the FEP that describes how agents explore and exploit their environment by minimising an expected FE objective. In two related papers, we describe a scalable, epistemic approach to synthetic AIF, by message passing on free-form Forney-style Factor Graphs (FFGs). A companion paper (part I) introduces a Constrained FFG (CFFG) notation that visually represents (generalised) FE objectives for AIF. The current paper (part II) derives message passing algorithms that minimise (generalised) FE objectives on a CFFG by variational calculus. A comparison between simulated Bethe and generalised FE agents illustrates how the message passing approach to synthetic AIF induces epistemic behaviour on a T-maze navigation task. Extension of the T-maze simulation to 1) learning goal statistics, and 2) a multi-agent bargaining setting, illustrate how this approach encourages reuse of nodes and updates in alternative settings. With a full message passing account of synthetic AIF agents, it becomes possible to derive and reuse message updates across models and move closer to industrial applications of synthetic AIF.

Figures (12)

• Figure 1: Example Forney-style factor graph (FFG) (left), constrained FFG (CFFG) (middle), and CFFG with indicated messages (right). The solid square indicates a clamped variable, and the solid circle a data constraint. Sum-product and variational messages are indicated by white and dark message circles respectively.
• Figure 2: A single slice of the Forney-style factor graph representation of the generative model of \ref{['eq:generative_ssm']}. State and parameter priors not drawn.
• Figure 3: Constrained Forney-style factor graph representations for variational objectives on models for past (left) and future states (right). The dashed box indicates a composite structure for the goal-observation submodel.
• Figure 4: CFFG of two facing nodes with indicated p-substitution and messages. Ellipses indicate an arbitrary number of adjacent edges (possibly zero), with beads indicating a joint variational distribution over the adjacent edges.
• Figure 5: Discrete-variable submodel with indicated constraints (left) and message updates (right), with $\sigma$ a softmax function. Updates for message two indicate the direct and indirect computation respectively.

Theorems & Definitions (12)

• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof