A Differentiable Partially Observable Generalized Linear Model with Forward-Backward Message Passing

TL;DR

The paper addresses learning in partially observable neural networks where hidden spikes must be inferred from observed data. It introduces a differentiable POGLM by relaxing hidden spike counts with a Gumbel-Softmax-based continuous surrogate, enabling pathwise gradient variational inference on $ELBO$ and wider reusable continuous distributions for $z_{t,h}$. A forward-backward message-passing sampling scheme for the variational model is proposed to better capture hidden-to-visible influences and improve posterior approximation. Across synthetic and real neural datasets, the method achieves higher test log-likelihoods, faster convergence, and more interpretable hidden-unit interactions, demonstrating practical value for neuroscience connectivity inference.

Abstract

The partially observable generalized linear model (POGLM) is a powerful tool for understanding neural connectivity under the assumption of existing hidden neurons. With spike trains only recorded from visible neurons, existing works use variational inference to learn POGLM meanwhile presenting the difficulty of learning this latent variable model. There are two main issues: (1) the sampled Poisson hidden spike count hinders the use of the pathwise gradient estimator in VI; and (2) the existing design of the variational model is neither expressive nor time-efficient, which further affects the performance. For (1), we propose a new differentiable POGLM, which enables the pathwise gradient estimator, better than the score function gradient estimator used in existing works. For (2), we propose the forward-backward message-passing sampling scheme for the variational model. Comprehensive experiments show that our differentiable POGLMs with our forward-backward message passing produce a better performance on one synthetic and two real-world datasets. Furthermore, our new method yields more interpretable parameters, underscoring its significance in neuroscience.

Figures (9)

• Figure 1: (a): The generative model of the complete POGLM $p(\bm X,\bm Z;\theta)$. (b), (c), (d): The forward-self, forward, and forward-backward sampling scheme of the variational model $q(\bm Z|\bm X;\phi)$.
• Figure 2: An example of comparison among different variational distributions $q(\bm Z|\bm X;\phi)$ with the true posterior $p(\bm Z|\bm X;\theta)$ (black dots). There is one visible neuron and one hidden neuron. Two visible spikes from the visible neuron happen at the dashed lines. Different dotted curves represent the approximated log-likelihood of one hidden spike happening at different time bins. Only the forward-backward recapitulates the true distribution. The forward and forward-self miss the uprising trends before the two observed spikes, due to lack of a back-propagated message.
• Figure 3: A visualization of different choices of the soft hidden spike count distribution, under firing rate $f=0.5$ and $f=1.0$. Most of $z$ from GS approximating the original Poisson distribution are close to integer points, but the three continuous distributions (Exp, Ray, and HN) are not.
• Figure 4: (a): The test log-likelihood (LL) on the test set, the weight error, the bias error, and the running time of different method combinations. (b): An example of the learned weight matrix and bias vector compared with the true of selected method combinations. Visualization of all method combinations is in Fig. \ref{['fig:synthetic_weight']} in Appendix. \ref{['appendix:supplementary_figures']}(c): The learning curves of different method combinations.
• Figure 5: The test log-likelihood (LL) of different method combinations under $H\in\left\{1,2,3\right\}$ hidden neurons. The dashed black line represents the test LL of the fully observed GLM as the baseline.