Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Brain in the Dark: Design Principles for Neuromimetic Inference under the Free Energy Principle

Mehran H. Bazargani, Szymon Urbas, Karl Friston

TL;DR

The paper tackles the weaknesses of deep learning—out-of-distribution generalisation, forgetting, and interpretability—by advocating neuromimetic AI built on the Free Energy Principle (FEP) and Predictive Coding. It provides a practical roadmap and a CPU-based PyTorch implementation to enable brain-like inference via variational methods, notably using Variational Free Energy $F$ as an objective. The authors formalise inference through $F(q;y^t)=D_{KL}[q(z^t)||p(z^t)]-\mathbb{E}_{q(z^t)}[\ln p(y^t|z^t)]$ and discuss discrete/continuous state-space formulations, including generalised coordinates of motion. A one-layer predictive coding experiment with Lotka-Volterra dynamics demonstrates improved state inference and model selection (e.g., $BF_{1,2}=1.36$ in favour of a nonlinear model $M_2$) and highlights the practical viability of FEP-inspired neuromimetic AI for robust perception tasks.

Abstract

Deep learning has revolutionised artificial intelligence (AI) by enabling automatic feature extraction and function approximation from raw data. However, it faces challenges such as a lack of out-of-distribution generalisation, catastrophic forgetting and poor interpretability. In contrast, biological neural networks, such as those in the human brain, do not suffer from these issues, inspiring AI researchers to explore neuromimetic deep learning, which aims to replicate brain mechanisms within AI models. A foundational theory for this approach is the Free Energy Principle (FEP), which despite its potential, is often considered too complex to understand and implement in AI as it requires an interdisciplinary understanding across a variety of fields. This paper seeks to demystify the FEP and provide a comprehensive framework for designing neuromimetic models with human-like perception capabilities. We present a roadmap for implementing these models and a Pytorch code repository for applying FEP in a predictive coding network.

Brain in the Dark: Design Principles for Neuromimetic Inference under the Free Energy Principle

TL;DR

The paper tackles the weaknesses of deep learning—out-of-distribution generalisation, forgetting, and interpretability—by advocating neuromimetic AI built on the Free Energy Principle (FEP) and Predictive Coding. It provides a practical roadmap and a CPU-based PyTorch implementation to enable brain-like inference via variational methods, notably using Variational Free Energy as an objective. The authors formalise inference through and discuss discrete/continuous state-space formulations, including generalised coordinates of motion. A one-layer predictive coding experiment with Lotka-Volterra dynamics demonstrates improved state inference and model selection (e.g., in favour of a nonlinear model ) and highlights the practical viability of FEP-inspired neuromimetic AI for robust perception tasks.

Abstract

Deep learning has revolutionised artificial intelligence (AI) by enabling automatic feature extraction and function approximation from raw data. However, it faces challenges such as a lack of out-of-distribution generalisation, catastrophic forgetting and poor interpretability. In contrast, biological neural networks, such as those in the human brain, do not suffer from these issues, inspiring AI researchers to explore neuromimetic deep learning, which aims to replicate brain mechanisms within AI models. A foundational theory for this approach is the Free Energy Principle (FEP), which despite its potential, is often considered too complex to understand and implement in AI as it requires an interdisciplinary understanding across a variety of fields. This paper seeks to demystify the FEP and provide a comprehensive framework for designing neuromimetic models with human-like perception capabilities. We present a roadmap for implementing these models and a Pytorch code repository for applying FEP in a predictive coding network.

Paper Structure

This paper contains 15 sections, 11 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Inference, learning, and uncertainty estimation
  3. Various problem formulations and their implications
  4. Predictive coding
  5. Experiments with a one-layer PC model & results
  6. Conclusion
  7. HMM for inference/learning
  8. A note on the generalised coordinates of motion
  9. The neuronal message passing in a one-layer PC network
  10. State inference pseudo-code
  11. The Lotka-Volterra GP and observations
  12. Variational free energy and variational Laplace
  13. Experiment further analysis
  14. The generative power of a one-layer PC network
  15. Calculating the gradients of the approximate VFE for a given generative model

Figures (4)

  • Figure 1: A Hidden Markov Model (HMM) for inference parr2022active.
  • Figure 2: Solution $x$, to the ODEs in the Lotka-Volterra GP (left) and the generated noisy observations $y$ (right).
  • Figure 3: Estimated hidden state and free action for $M_1$ in top panel (left) and bottom panel (left), respectively, and estimated hidden state and free action for $M_2$ in top panel (right) and bottom panel (right), respectively.
  • Figure 4: Predicted sensations and actual sensations for model $M_1$ (top panel) and model $M_2$ (bottom panel).