A Concise Mathematical Description of Active Inference in Discrete Time
Jesse van Oostrum, Carlotta Langer, Nihat Ay
TL;DR
This work provides a concise, mathematically precise treatment of active inference in discrete time, formalizing the agent's generative model as $p(o_{1:T}, s_{1:T}| a_{1:T-1}, \theta)$ and the approximate belief state as $q_t(\cdot)$. Action selection is implemented by drawing a policy from a softmax over the negative expected free energy $G(\pi_t| o_{1:t}, a_{1:t-1})$, with two equivalent decompositions of $G$ into epistemic value and utility (or ambiguity and risk), plus mean-field approximations for tractability. The paper also details learning via Dirichlet priors over model parameters $\theta=(\theta^D,\theta^A,\theta^B)$, with update rules $\alpha'^D_j=\alpha^D_j + q_T(s_1^{(j)})$, $\alpha'^A_{ij}=\alpha^A_{ij} + \sum_{\tau=1}^T \mathbb{1}_{o^{(i)}}(o_\tau) q_T(s_\tau^{(j)})$, and $\alpha'^B_{jkl}=\alpha^B_{jkl} + \sum_{\tau=2}^T q_T(s_\tau^{(j)}) q_T(s_{\tau-1}^{(k)}) \mathbb{1}_{a^{(l)}}(a_{\tau-1})$, with posterior means $\hat{\theta}$ used for model updates. An illustrative T-maze example demonstrates state inference and policy selection in a finite-horizon setting, making the core machinery readily implementable and extensible to pymdp environments. The contributions provide a clear, interoperable framework for discrete-time active inference with explicit equations and a lightweight Python reference implementation.
Abstract
In this paper we present a concise mathematical description of active inference in discrete time. The main part of the paper serves as a basic introduction to the topic, including a detailed example of the action selection mechanism. The appendix discusses the more subtle mathematical details, targeting readers who have already studied the active inference literature but struggle to make sense of the mathematical details and derivations. Throughout, we emphasize precise and standard mathematical notation, ensuring consistency with existing texts and linking all equations to widely used references on active inference. Additionally, we provide Python code that implements the action selection and learning mechanisms described in this paper and is compatible with pymdp environments.