Efficient and robust control with spikes that constrain free energy

TL;DR

This work provides a novel mathematical account for spiking control through constraining free energy, providing both better insight into how brain networks might leverage their spiking substrate and a new route for implementing efficient control algorithms in neuromorphic hardware.

Abstract

Animal brains exhibit remarkable efficiency in perception and action, while being robust to both external and internal perturbations. The means by which brains accomplish this remains, for now, poorly understood, hindering our understanding of animal and human cognition, as well as our own implementation of efficient algorithms for control of dynamical systems.A potential candidate for a robust mechanism of state estimation and action computation is the free energy principle, but existing implementations of this principle have largely relied on conventional, biologically implausible approaches without spikes. We propose a novel, efficient, and robust spiking control framework with realistic biological characteristics. The resulting networks function as free energy constrainers, in which neurons only fire if they reduce the free energy of their internal representation. The networks offer efficient operation through highly sparse activity while matching performance with other similar spiking frameworks, and have high resilience against both external (e.g. sensory noise or collisions) and internal perturbations (e.g. synaptic noise and delays or neuron silencing) that such a network would be faced with when deployed by either an organism or an engineer. Overall, our work provides a novel mathematical account for spiking control through constraining free energy, providing both better insight into how brain networks might leverage their spiking substrate and a new route for implementing efficient control algorithms in neuromorphic hardware.

Figures (7)

• Figure 1: Control Loop of Spiking Free Energy Constrainer (SFEC) networks, displayed schematically, for controlling a single Spring-Mass-Damper linear dynamical system. A) The network receives the system observations ($\mathbf y$) and a target state ($\mathbf z$) as input, as well as recurrent connections of two different timescales ($\mathbf \Omega$), and outputs sequences of spikes that drive the network's internal representation. B) The network's objective is to maintain a bounded free energy, which is computed from the precision-weighed prediction errors ($F$, top). This corresponds to a quadratic optimization process that maintains low prediction errors geirhos_shortcut_2020mancoo_understanding_2020. Spikes are generated from this process during states with high free energy, as we can see by the fact that activity in the spike raster (bottom) is concentrated around the moments when the target changes instantaneously. C) SFEC networks maintain an internal estimate ($\boldsymbol \mu$), which is driven by the spikes. The weights of the network are arranged such that the neurons' thresholds form a bounding box calaim_geometry_2022 around the point of minimal free energy and each neuron's firing will return the signal closer to this point. The gray polygon within the box indicates the limits of where the internal errors are constrained to be ($\boldsymbol \epsilon_{\boldsymbol{\mu}}$ and $\boldsymbol \epsilon_{\boldsymbol{y}}$), thus constraining the free energy. When either error becomes to big, thus "hitting" one of the colored edges, the corresponding neuron fires, moving the errors along the vector $\boldsymbol{D_i}$ towards the center of the bounding box. The internal estimate is updated such that the free energy ($F$) remains bounded through a bounding box mechanism calaim_geometry_2022. Because the free energy combines measures of the error with the measured state and the target state, the internal estimate remains between the two. D) Based on the internal estimate (which is biased towards the target state) a control action (force) is generated and applied to the system driving it from its actual state towards the internal state, ensuring it follows the target dynamics. E) The state variables evolve according to the Plant dynamics, the control input and a process noise term. The system used in this example is the linear SMD system. F) Observations are generated from the current state of the system and an observation noise term, and fed back into the network.
• Figure 2: SFEC can successfully control coupled dynamical systems. A) (i) Schematic of a Coupled Oscillator system with 5 masses and 6 springs. The masses oscillate between two walls with fixed positions at $x=0$ and $x=L$. (ii) Control of a coupled Oscillator system showing the oscillator states converging to the target states (top), the control signals generated by the control algorithm (middle), and the spike raster showcasing network activity (bottom). B) (i) Schematic of a system of drones (modelled as 2D free masses). The green lines represent the attractive forces the controller exerts towards the target positions, the big X's are those target positions and the dotted arrows are the trajectory the target signal is following (ii) Same as panel A (ii) but for control of a drone swarm system.
• Figure 3: Flexible target dynamics enable different coordination behaviors. A) (i) Simple target dynamics: each mass is independently attracted to a target position (illustrated by springs from targets to drones, as illustrated by the green dotted lines); [explain green lines, arrows, X's]. (ii) 2D trajectories of three drones following independent attractors — each drone moves directly to its target. B) (i) Formation target dynamics: masses are attracted to targets while repelling each other (green lines show attraction, red lines show repulsion). (ii) 2D trajectories showing formation maintenance — drones coordinate their motion to avoid collisions while following a central target which moves on a circle (dashed line).
• Figure 4: SFEC robustness to cumulative perturbations. (Top row) A timeline indicating when each perturbation is activated). (Rows 2-4) Target and state changes, spike raster, free energy, and the MSE. A) Control robustness under external perturbations, consisting of random kicks (throughout), increased process noise ($t \geq 10s$) and increased observation noise ($t \geq 20s$). B) Control robustness under internal perturbations applied to the controller network, consisting of voltage noise ($t \geq 6s$), synaptic perturbations ($t \geq 12s$), 25% neuron silencing ($t \geq 18s$), synaptic delays ($t \geq 24s$).
• Figure 5: Extensive analysis of the controller's robustness to noise. A) Heat Map plot showing Control Noise in the x-axis and Observation Noise in the y-axis; total MSE of the entire control run ($T=30s, dt=0.001s$, 2D masses) is shown as color, where brighter color represents more error. B) Example of a run with minimal Process Noise and very high Observation Noise, corresponding to the highlighted box on the top left. C) Example of a run with very high Process Noise and minimal Observation Noise, corresponding to the highlighted box on the bottom right. In both B and C, the top subplot shows the real positions of the masses, whilst the bottom subplot shows the observations fed to the controller.