Predictive Learning in Energy-based Models with Attractor Structures

TL;DR

This work addresses how the brain predicts future observations after actions by proposing a biologically plausible, hierarchical energy-based model (EBM) with a continuous attractor neural network (CANN) memory. It advances a recurrent state-space formulation where learning is local and Hebbian-like, inference uses sampling-based posterior updates, and memory is stored in a CANN, avoiding backpropagation through time. Empirical results across diverse visual tasks show accurate predictions on trained environments and reasonable generalization to unseen ones, with competitive performance to machine-learning baselines. The approach offers a potentially scalable, online world-modeling framework with implications for biologically plausible predictive processing and future reinforcement learning integration.

Abstract

Predictive models are highly advanced in understanding the mechanisms of brain function. Recent advances in machine learning further underscore the power of prediction for optimal representation in learning. However, there remains a gap in creating a biologically plausible model that explains how the neural system achieves prediction. In this paper, we introduce a framework that employs an energy-based model (EBM) to capture the nuanced processes of predicting observation after action within the neural system, encompassing prediction, learning, and inference. We implement the EBM with a hierarchical structure and integrate a continuous attractor neural network for memory, constructing a biologically plausible model. In experimental evaluations, our model demonstrates efficacy across diverse scenarios. The range of actions includes eye movement, motion in environments, head turning, and static observation while the environment changes. Our model not only makes accurate predictions for environments it was trained on, but also provides reasonable predictions for unseen environments, matching the performances of machine learning methods in multiple tasks. We hope that this study contributes to a deep understanding of how the neural system performs prediction.

Figures (8)

• Figure 1: (a) The directed graphical model of the generative model. Taking $t=2$ as an example, $s_2$ follows the prior, $o_2$ follows the likelihood function, and $m_3$ follows the transition probability. (b) Taking $t=2$ as an example, the brain initially generates a prediction $\hat{s}_2$ for the neural activity, followed by producing a prediction $\hat{o}_2$ for the observation. Then the network parameters are updated, as indicated by the dashed lines, and the posterior for the current time step is obtained. At last, the memory is updated based on action $a_3$ and the sample $s_2$ following the posterior.
• Figure 2: General process
• Figure 3: (a) In the hierarchical structure, the activity of neurons in the upper layer is the observation for the neurons in the lower layer. In the case of $L=2$, $s^0_t$ is the observation of $s^1_t$, and $s^1_t$ is the observation of $s^2_t$. The bottom-left box illustrates the connection of the memory-representing CANN with action neurons and its link to $s^2$ through $e ^2$. The top-right box shows the connections between $s^0$ and $s^1$ through $e^0$. (b) Based on the current memory $m_1$, we imagine the observations we would receive at each time step after taking a series of actions.
• Figure 4: Hierarchical neural process
• Figure 5: Experiments on modeling eye movement. (a) Generation results of the entire image through initialized memory. The orange and blue areas in the first column indicate the patches used for initialization and those that need to be predicted. 'Seen' refers to images that were used during training, while 'unseen' refers to images that were not used during training (here, both are the same images because different models were used). (b) Testing phase: the first step is initialization, similar to the training process, involves executing actions, observing the environment, inferring the latent state, and finally updating the memory. However, unlike training, network weights are not updated in this step. The second step involves executing random actions and predicting observations (c) Model tested by memory initialized with $K = 4$ patches. (d) Model trained on $N=32$ images. When initialized with $K=6$ patches , the whole image can be almost completely reconstructed. (e) The $x$-axis neuron numbers represent the number of neurons in each layer, with a total of $L=3$ layers in the network.