Expected Free Energy-based Planning as Variational Inference
Bert de Vries, Wouter Nuijten, Thijs van de Laar, Wouter Kouw, Sepideh Adamiat, Tim Nisslbeck, Mykola Lukashchuk, Hoang Minh Huu Nguyen, Marco Hidalgo Araya, Raphael Tresor, Thijs Jenneskens, Ivana Nikoloska, Raaja Ganapathy Subramanian, Bart van Erp, Dmitry Bagaev, Albert Podusenko
TL;DR
This work unifies planning under uncertainty with variational inference by showing that Expected Free Energy (EFE) minimization can be derived from a variational free energy on a generative model augmented with preference and epistemic priors. The central result, the Expected Free Energy Theorem, yields a decomposition $F[q] = E_{q(u)}[ G(u) ] + E_{q(yx\theta u)}[\log \frac{q(yx\theta u)}{p(yx\theta u)}]$, and leads to a Bayes-optimal, resource-aware policy update $q^*(u) = \sigma(-P(u) - G(u) - C(u))$. Epistemic priors bias the agent toward ambiguity reduction and information gain, while the complexity term $C(u)$ encodes inference constraints under bounded resources. The framework supports scalable, interruptible, distributed planning via reactive message passing on factor graphs, offering a principled route to synthetic active inference agents that integrate goal attainment with epistemic exploration. Overall, the paper strengthens the theoretical foundations of active inference and provides a path toward practical, scalable planning under uncertainty.
Abstract
We address the problem of planning under uncertainty, where an agent must choose actions that not only achieve desired outcomes but also reduce uncertainty. Traditional methods often treat exploration and exploitation as separate objectives, lacking a unified inferential foundation. Active inference, grounded in the Free Energy Principle, provides such a foundation by minimizing Expected Free Energy (EFE), a cost function that combines utility with epistemic drives, such as ambiguity resolution and novelty seeking. However, the computational burden of EFE minimization had remained a significant obstacle to its scalability. In this paper, we show that EFE-based planning arises naturally from minimizing a variational free energy functional on a generative model augmented with preference and epistemic priors. This result reinforces theoretical consistency with the Free Energy Principle by casting planning under uncertainty itself as a form of variational inference. Our formulation yields policies that jointly support goal achievement and information gain, while incorporating a complexity term that accounts for bounded computational resources. This unifying framework connects and extends existing methods, enabling scalable, resource-aware implementations of active inference agents.