A Message Passing Realization of Expected Free Energy Minimization

TL;DR

The paper addresses planning under epistemic uncertainty by reframing Expected Free Energy minimization as Variational Free Energy minimization on factor graphs. It introduces a message-passing algorithm that augments the generative model with epistemic priors, turning a combinatorial search into tractable inference via standard VI techniques. Empirical evaluation in a stochastic gridworld and a partially observable Minigrid task shows that EFE-minimizing agents outperform KL-control agents, exhibiting risk-averse behavior in uncertain dynamics and systematic information-seeking under partial observability. This work provides a scalable bridge between active inference theory and practical decision-making, delivering a principled framework for balancing pragmatic goals with epistemic exploration in complex environments.

Abstract

We present a message passing approach to Expected Free Energy (EFE) minimization on factor graphs, based on the theory introduced in arXiv:2504.14898. By reformulating EFE minimization as Variational Free Energy minimization with epistemic priors, we transform a combinatorial search problem into a tractable inference problem solvable through standard variational techniques. Applying our message passing method to factorized state-space models enables efficient policy inference. We evaluate our method on environments with epistemic uncertainty: a stochastic gridworld and a partially observable Minigrid task. Agents using our approach consistently outperform conventional KL-control agents on these tasks, showing more robust planning and efficient exploration under uncertainty. In the stochastic gridworld environment, EFE-minimizing agents avoid risky paths, while in the partially observable minigrid setting, they conduct more systematic information-seeking. This approach bridges active inference theory with practical implementations, providing empirical evidence for the efficiency of epistemic priors in artificial agents.

Figures (10)

• Figure 1: A Forney-style factor graph representation of the factorization in \ref{['eq:ffg:examplefactorization']}.
• Figure 2: Slice of the factor graph representation of the augmented generative model. The original generative model \ref{['eq:factorized-model']} is augmented with epistemic priors $\tilde{p}(u_{t+1})$ and $\tilde{p}(x_{t+1})$, and preference priors $\hat{p}(x_{t+1})$ for future timesteps.
• Figure 3: The stochastic grid environment. The agent should traverse the grid with both stochastic transitions and observation noise. Cells with stochastic transitions appear on the shortest path, creating a risk-reward tradeoff. Opacity for observation noise is used to indicate the uncertainty in the environment.
• Figure 4: An initial state of the Minigrid environment. The agent has a limited field of view, indicated by the highlighted cells.
• Figure 5: Visualization of the inference results for the stochastic grid environment. On the left, the initial state of the environment is shown. On the right we show the Bethe Free Energy curve over the iterations of message passing. Convergence to a constant value indicates convergence of the inference procedure.

Theorems & Definitions (5)

• theorem \@thmcountertheorem: Expected Free Energy Theorem
• proof
• corollary \@thmcountercorollary
• proof
• lemma \@thmcounterlemma: Proof of equivalence $C(\bm{u})$ in \ref{['eq:F-C-1']} and \ref{['eq:F-C-2']}