An Active Inference perspective on Neurofeedback Training

TL;DR

This paper tackles the problem of highly variable NFT outcomes by introducing an active inference (AIF) model of the NFT closed loop. By formulating NFT as a POMDP where perception, action, and learning minimize variational and expected free energy, the authors simulate agents with subject-specific priors and biomarkers to predict training trajectories under noise, instruction, and interoceptive cues. Key findings show that feedback quality, prior beliefs, and internal signals jointly govern learning success; perfect feedback alone does not guarantee regulation. The work offers a principled, testable framework to predict NFT variability and guides the design of personalized NFT protocols, including the potential importance of interoceptive feedback for generalization. Overall, the AIF-based approach provides a robust, mechanistic lens to interpret NFT data and optimize training protocols for diverse users.

Abstract

Neurofeedback training (NFT) aims to teach self-regulation of brain activity through real-time feedback, but suffers from highly variable outcomes and poorly understood mechanisms, hampering its validation. To address these issues, we propose a formal computational model of the NFT closed loop. Using Active Inference, a Bayesian framework modelling perception, action, and learning, we simulate agents interacting with an NFT environment. This enables us to test the impact of design choices (e.g., feedback quality, biomarker validity) and subject factors (e.g., prior beliefs) on training. Simulations show that training effectiveness is sensitive to feedback noise or bias, and to prior beliefs (highlighting the importance of guiding instructions), but also reveal that perfect feedback is insufficient to guarantee high performance. This approach provides a tool for assessing and predicting NFT variability, interpret empirical data, and potentially develop personalized training protocols.

Figures (14)

• Figure 1: A schematic representation of the neurofeedback closed-loop paradigm. 1. The subject physiological brain signals (electromagnetic, BOLD signal, etc.) are acquired using EEG, MEG, fMRI, etc.2. They are used to infer the hidden subject cognitive state that caused it. The experimenter derives a task-specific feedback from the infered state. Note that this latent state inference relies on strong hypotheses made by the experimenter about parts of the neurophysiological / measuring process underlying the neurofeedback. 3. The subject perceives the feedback through various sensory means. (auditory, visual, etc.) and tries to relate it to its own cognitive state (4.).The subject thus tries to learn the relationship between his hidden cognitive states and the indicator displayed on the screen. On the left, a few other factors which may influence the cognitive dynamics of the subject training are shown.
• Figure 2: Generative process and generative model : the subject's brain models the neurofeedback environment to predict outcomes and optimize its actions. Remarkably, BCI training casts cognitive states as external "environment" variables and includes them in its generative model. It is thus necessary to make a distinction between the hidden states themselves and how they are perceived internally by the subject. The agent can only interact with its environment through boundary states (its sensory and active states).
• Figure 3: The Partially Observable Markov Decision Process (POMDP) used to model training. The environment (generative) process figures a set of hidden states ($\mathbf{\hat{s}_t}$), corresponding to the subject "actual" cognitive states in our formulation. Although impossible to see directly, each state stochastically generates stimuli observable by the subject ($o_t$) depending on a true observation function $\mathbf{A}$. It may consist in some external feedback or in an interoceptive observer the subject must learn to interpret. Possible state transitions ($\mathbf{B}$) and starting states ($\mathbf{D}$) are fixed before training. The subject generative model of the cognitive regulation is shown above, featuring a set of perceived states ($\mathbf{s_t}$). The subject models the feedback as a realization of hidden states with the function $\mathbf{a}$, and the effect of his/her mental actions $u_t$ on those states with the function $\mathbf{b}$. The Active Inference agent uses this formulation to update their model on two distinct timescales by minimizing their Free Energy following the equations described in sophisticated_inferenceda_costa_active_2020. On the timestep timescale, it infers the hidden states that best match observations and prior beliefs, and the actions that best match its habits/preferences/exploration drive. On the trial timescale, the agent updates its beliefs about the environment dynamics ($\mathbf{a}$,$\mathbf{b}$,$\mathbf{d}$) to better predict and navigate it.
• Figure 4: The Active Inference framework unifies agent perception, action and learning by iteratively minimizing these functions across different timescales.
• Figure 5: Full model. The environment (generative process) forwards observations to the subject as a discrete feedback value (here $o_t=4$) and reacts to subject actions $u_t \sim \Pi_t$. In our simulations, the process state topography was a simple graph with 5 possible states and 5 possible ooutcomes. Possible state transitions are showed with blue arrows ($\mathbf{B}$) and starting state with red arrows ($\mathbf{D}$). Each state may generates outcomes based on the feedback mapping ($\mathbf{A}$) following a normal law $N(\mu,\sigma_{process})$ (unbiaised noisy feedback). Initially, the agent believes each state is correlated with a specific feedback level following a normal law $N(\mu,\sigma_{model})$ (unbiaised noisy feedback). Given these priors, the agent attempts to learn state transitions and starting values ($\mathbf{b}$ and $\mathbf{d}$) in order to achieve higher feedback levels. This is done in 3 steps : the subject uses its priors to 1. infer the hidden states, 2. pick the best actions (for exploratory or exploitative purposes) and finally 3. update its mapping from the observed succession of actions/observations.