Spiking neurons as predictive controllers of linear systems

Abstract

Neurons communicate with downstream systems via sparse and incredibly brief electrical pulses, or spikes. Using these events, they control various targets such as neuromuscular units, neurosecretory systems, and other neurons in connected circuits. This gave rise to the idea of spiking neurons as controllers, in which spikes are the control signal. Using instantaneous events directly as the control inputs, also called `impulse control', is challenging as it does not scale well to larger networks and has low analytical tractability. Therefore, current spiking control usually relies on filtering the spike signal to approximate analog control. This ultimately means spiking neural networks (SNNs) have to output a continuous control signal, necessitating continuous energy input into downstream systems. Here, we circumvent the need for rate-based representations, providing a scalable method for task-specific spiking control with sparse neural activity. In doing so, we take inspiration from both optimal control and neuroscience theory, and define a spiking rule where spikes are only emitted if they bring a dynamical system closer to a target. From this principle, we derive the required connectivity for an SNN, and show that it can successfully control linear systems. We show that for physically constrained systems, predictive control is required, and the control signal ends up exploiting the passive dynamics of the downstream system to reach a target. Finally, we show that the control method scales to both high-dimensional networks and systems. Importantly, in all cases, we maintain a closed-form mathematical derivation of the network connectivity, the network dynamics and the control objective. This work advances the understanding of SNNs as biologically-inspired controllers, providing insight into how real neurons could exert control, and enabling applications in neuromorphic hardware design.

Figures (11)

• Figure 1: Overview of feedback control approaches. A: Scheme of a feedback controller and a downstream system. The current state of the system $\mathbf{x}\mathit{(t)}$ is given as input to the controller, along with a target value $\mathbf{z}\mathit{(t)}$. The controller outputs a control signal $\mathbf{u}\mathit{(t)}$, as a function of both inputs. $\mathbf{u}\mathit{(t)}$ is then mapped onto the dynamics of the downstream system via the $\mathbf{B}$ matrix. The system's dynamics are governed by the $\mathbf{A}$ matrix. B-D: Examples of three feedback control paradigms. B: The continuous LQR uses a gain matrix $\mathbf{K}$ to directly compute the control signal for a given state and target pair. C: In the filtered spiking case, the output spikes of the network are filtered to obtain $\mathbf{r}\mathit{(t)}$, which is decoded by the matrix $\mathbf{D}$ to form the control signal $\mathbf{u}\mathit{(t)}$ (adapted from slijkhuisClosedform2023). D: In our spiking control paradigm, the output spikes $\mathbf{s}\mathit{(t)}$ represent the control input itself, and are therefore directly mapped onto the system dynamics through the $\mathbf{B}$ matrix.
• Figure 2: Illustration of the network-system loop. Scheme of a recurrent spiking network composed of four neurons controlling a downstream system. Inputs and target are mapped onto the network via forward connections. $\mathbf{G}$ maps the target $\mathbf{z}$, while $\mathbf{F}$ maps the system's current state, $\mathbf{x}$. The matrix $\mathbf{\Omega}$ represents the recurrent weights. The control inputs are the spikes $\mathbf{s}$, which are mapped onto the system via the matrix $\mathbf{B}$.
• Figure 3: Examples of reactive and predictive spiking. A: Example of a linear dynamical system with its current state (brown dot), and four example states that can be reached with a spike. The gray dot represents a target state. We employ a reactive spiking rule. The neuron for which $L^\text{s}_i < L^\text{ns}$ spikes (green arrow), since that immediately shortens the distance between the system's state and the target. The other ones do not (red arrows). B: Example of a linear dynamical system, where the spike decision is taken based on the state of the system $f$ seconds after the spike. In this predictive case, $L^\text{s}_i < L^\text{ns}$ is still the spiking condition, but the preferred spiking direction can differ. The condition $L^\text{s}_i<L^\text{ns}$ holds for both the upward (green) and rightward (orange) spikes, since the no-spike trajectory (brown) lies further from the target (gray dot) than either of the spiking ones. Here, an asynchronous firing rule (see Results) would select only one of these neurons to fire; either at random, or by considering the neuron with the highest voltage (green).
• Figure 4: Spiking control simulations. The two rows denote reactive and predictive systems. The two columns represent a generic linear system, and the physical SMD. All the examples simulation are run with the simplest network possible of two neurons. A: On the left side, state space representation of a reactive spiking control approach for a generic unconstrained system. The gray line corresponds to the changing target. Although the matrix $\mathbf{B}$ allows for spikes in all four directions, the network continuously spikes rightwards, ignoring the intrinsic dynamics of the plant. On the right side, the top figure shows the changes of velocity, position and target over time, while the bottom figure shows the spike control input (each spike scaled by corresponding value in the matrix $\mathbf{B}$). B: On the left side, the state space figure shows that without predictiveness, a physically constrained system cannot be controlled using our greedy spiking rule. In this example, $\mathbf{B}$ is constrained to only allow spikes on the velocity component, which do not cause any immediate effect on the position, preventing the loss to be decreased by the spike events. When controlling a physical system with a reactive control approach, the loss in the non-spiking case is always smaller than the loss in the spiking case, and the network never spikes. On the right side, no activity is present. C: In the predictive, unconstrained example, the system can spike in all four directions but considers its future state. In this case, the network either spikes directly towards the target or spikes upwards, exploiting the systems dynamics. D: Use of our predictive SNN approach for the control of an SMD. As shown in the state space representation, as the target position changes, the network spikes up or down, adjusting the velocity of the plant based on its predicted state in $f$ seconds. On the right side, we highlight how the network's activity adjusts to the target. It employs downwards spikes to adjust for an eventual overshooting of the position relative to the target, and produces consecutive upwards spikes when the target is changing more rapidly.
• Figure 5: Example activity of a network of four neurons. Each spike plotted as control signal is modeled as a Dirac delta function that gets mapped onto the plant's dynamics through the matrix $\mathbf{B}$, and instantaneously influences the velocity component of the system (see Results). Spike moments and the relative voltage traces are shown, as well as the resulting errors. The error trace (deep blue line) describes the current distance between the position and a target state, while the predictive error (light blue line) measures the distance between the target and a predicted position, $f$ seconds into the future. At time $T=20$ seconds into the simulation, we inject noise (with standard deviation $\sigma=0.08$) in the voltages of the neurons. The network remains robust to noisy spiking and the control performance is not affected.