Spike-Timing-Dependent Plasticity for Bernoulli Message Passing

TL;DR

This work addresses implementing Bayesian inference on spike-based hardware by training spiking neural networks to realize sum-product message passing for Bernoulli distributions $p_x$, $p_y$, and $p_z$. It uses spike-timing-dependent plasticity (STDP) in networks of Leaky Integrate-and-Fire (LIF) neurons to implement factor nodes (e.g., AND/OR/NOT/XOR) within Forney-style Factor Graphs, encoding Bernoulli messages into spike trains and decoding outputs back to Bernoulli distributions via $p_z = \rho/\tau$. The results show that the SNN-based messages closely match numerical sum-product updates, demonstrated on an unreliable binary channel example, illustrating the approach's generality. This work provides a biologically plausible pathway to energy-efficient probabilistic inference on neuromorphic hardware, bridging the Bayesian brain/FEP perspective with spike-based computation.

Abstract

Bayesian inference provides a principled framework for understanding brain function, while neural activity in the brain is inherently spike-based. This paper bridges these two perspectives by designing spiking neural networks that simulate Bayesian inference through message passing for Bernoulli messages. To train the networks, we employ spike-timing-dependent plasticity, a biologically plausible mechanism for synaptic plasticity which is based on the Hebbian rule. Our results demonstrate that the network's performance closely matches the true numerical solution. We further demonstrate the versatility of our approach by implementing a factor graph example from coding theory, illustrating signal transmission over an unreliable channel.

Figures (6)

• Figure 1: The FFG corresponding to the example model defined in Equation \ref{['eq:Model-example1']}.
• Figure 2: SNN architecture for simulating the sum-product update rules of AND, OR, and NOT factor nodes, as represented in Table \ref{['table:mp-logic']}. Random variables $X$ and $Y$ are encoded into spike trains, and the resulting output spike train is decoded into $Z$. Each circle represents a LIF neuron. The training layer is removed after training the synaptic weights $\omega_1$ and $\omega_2$ using the STDP algorithm. For computing NOT of $X$, the variable $Y$ is configured to emit constant spikes at every time step by defining its Bernoulli distribution as $\overrightarrow{\mu}_y(y) = \mathcal{B}er(y \mid p_y = 1)$.
• Figure 3: Comparison of the results obtained from the proposed SNN-based nodes for passing Bernoulli messages with those produced by the sum-product algorithm. Validation was performed using eight random pairs of $p_x$ and $p_y$.
• Figure 4: Evolution of synaptic weights for AND, OR, and NOT nodes, from STDP-based training described in Section \ref{['sec: method']}.
• Figure 5: SNN architecture for simulating the sum-product update rule of XOR factor node as defined in Table \ref{['table:mp-b']}. The component networks reused from the AND, OR, and NOT implementations are shown in Figure \ref{['fig: logical_gates1']}, and the same encoding and decoding methods are applied.