Inferring stochastic low-rank recurrent neural networks from neural data

TL;DR

The paper addresses inferring stochastic, low-rank recurrent neural networks (RNNs) from noisy neural data by formulating the dynamics in a reduced $R$-dimensional latent space using a rank-$R$ connectivity $ extbf{J}= extbf{M} extbf{N}^{\top}$ and mapping back to observed activity. It introduces variational sequential Monte Carlo (SMC) to fit these models, including an encoder-based proposal for nonlinear observations and a Generalised Teacher Forcing mechanism to stabilize temporal inference. Empirically, the approach yields lower-dimensional latent dynamics across EEG, hippocampal spiking, and monkey-reaching data while providing a tractable fixed-point analysis for piecewise-linear activations; it also demonstrates favorable comparisons to state-of-the-art methods in reconstruction quality with smaller latent dimensionality. The work thus offers a principled, generative framework that captures trial-to-trial neural variability with interpretable, analytically tractable dynamics, and it outlines concrete directions for extending to broader noise models and multi-modal neural data.

Abstract

A central aim in computational neuroscience is to relate the activity of large populations of neurons to an underlying dynamical system. Models of these neural dynamics should ideally be both interpretable and fit the observed data well. Low-rank recurrent neural networks (RNNs) exhibit such interpretability by having tractable dynamics. However, it is unclear how to best fit low-rank RNNs to data consisting of noisy observations of an underlying stochastic system. Here, we propose to fit stochastic low-rank RNNs with variational sequential Monte Carlo methods. We validate our method on several datasets consisting of both continuous and spiking neural data, where we obtain lower dimensional latent dynamics than current state of the art methods. Additionally, for low-rank models with piecewise linear nonlinearities, we show how to efficiently identify all fixed points in polynomial rather than exponential cost in the number of units, making analysis of the inferred dynamics tractable for large RNNs. Our method both elucidates the dynamical systems underlying experimental recordings and provides a generative model whose trajectories match observed variability.

Figures (21)

• Figure 1: Our goal is to obtain generative models from which we can sample realistic neural data while having a tractable underlying dynamical system. We achieve this by fitting stochastic low-rank RNNs with variational sequential Monte Carlo.
• Figure 1: Proof sketch including $\mathbf{D}_{\Omega}$'s. The phase-space of an RNN with $N$ (here 2) units with activation $\max(0,\mathbf{x}_i-\mathbf{h}_i)$ is partitioned into $2^N$ (here 4) regions in which the dynamics are linear, each corresponding to a configuration of $\mathbf{D}_{\Omega}$. If dynamics are confined to the $R$-dimensional subspace spanned by the columns of $\mathbf{M}$, only a subset (here 3) can be reached. Each unit intersects the space spanned by the columns of $\mathbf{M}$ with a hyperplane (the pink points in the Figure). The amount of linear regions in $\mathbf{M}$, thus becomes equivalent to "how many regions can we create in $R$-dimensional space with $N$ hyperplanes?"
• Figure 2: Proof sketch.
• Figure 2: a) An arrangement of 3 hyperplanes $a_1,a_2$ and $a_3$ in general position. b) the associated intersection poset of the arrangement in a. c) An alternative arrangement with its associated intersection poset. d). Blue numbers indicate the value of the Möbius function.
• Figure 3: RNNs recover dynamics in teacher-student setups. a) Example ground truth latent trajectory and phase plane of low-rank RNN trained to oscillate (top left) and noisy observations of neuron activity (top right; 6/20 shown). A second low-rank RNN trained on the activity of the first recovers ground truth dynamics. b) Same set-up, but with Poisson observations. c) The teacher network was trained on a task where it has to provide an output corresponding to 8 different angles depending on an input cue. The student network, when given the same input during fitting, recovers the approximate ring attractor. d) Mean ($\pm1$SD) autocorrelation of the latents of the models from panel a, show the oscillation frequency is captured, as well as the decorrelation due to recurrent noise. The scale of the observed rates also agrees between student and teacher. e) Mean rates and ISI between student and teacher units of panel b match. f) Example rate distribution of one unit of the teacher and student RNN (of panel c), after onset of the 8 different stimuli.

Theorems & Definitions (7)

• Theorem 1: Zaslavsky's Theorem; Zaslavsky1975Stanley2007
• Lemma 1
• proof
• Lemma 2
• proof
• Proposition 1
• proof