Space as Time Through Neuron Position Learning

TL;DR

This work introduces neuron position learning, where inter-neuron delays are set by Euclidean distance, yielding space-time coupled spiking networks. The authors derive gradients for neuron positions and show that distance-based delays act as a meaningful inductive bias, guiding the network toward local, modular, small-world topologies during temporal classification on SHD. Spatial embeddings reproduce many benefits of learnable delays while promoting locality and functional specialization without explicit enforcement. The findings offer mechanistic interpretability, potential neuromorphic advantages, and a pathway to biologically inspired models that tightly couple spatial layout with temporal dynamics.

Abstract

Biological neural networks exist in physical space where distance determines communication delays: a fundamental space-time coupling absent in most artificial neural networks. While recent work has separately explored spatial embeddings and learnable synaptic delays in spiking neural networks, we unify these approaches through a novel neuron position learning algorithm where delays relate to the Euclidean distances between neurons. We derive gradients with respect to neuron positions and demonstrate that this biologically-motivated constraint acts as an inductive bias: networks trained on temporal classification tasks spontaneously self-organize into local, small-world topologies with modular structure emerging under distance-dependent connection costs. Remarkably, we observe unprompted functional specialization aligned with spatial clustering without explictly enforcing it. These findings lay the groundwork for networks in which space and time are intrinsically coupled, offering new avenues for mechanistic interpretability, biologically inspired modelling, and efficient implementations.

Figures (4)

• Figure 1: A Input example from the SHD dataset. B Example of preferred positions of input neurons based on the learnt synaptic strengths. C Example of preferred position of input neurons, based on their activity. D Examlpe of preferred position over time. The depicted plots are from networks trained with L1+distance cost regularisation, but we observe such spatial patterns under all training circumstances.
• Figure 2: Network shapes without (A) and with spatial cost (B). Line thickness corresponds to synaptic strength. Neurons move close to the perimeter in both cases, but when introducing the distance cost, we have fewer long strong connections. C-F Training dynamics over epochs. D shows that the network with the introduced cost term achieves a higher modularity than the other two. In E we observe that spatial networks achieve a lower Shannon entropy in their weights, regardless of the cost term. On C we see that if calculate the Shannon entropy on the communicability matrix derived from the weight matrix, the network with the distance cost has a lower value. On C we observe that introducing the cost term yields higher small-worldness.
• Figure 3: A pruning effects on classification accuracy on SHD. Pruning is less harmful for the position learning network. B delay distributions post training. For the non-spatial network, most delays are still zero after training, but we also end up with a few very long delays that are non visible since they are significantly outnumbered by the zero delays. For the spatial network delays become spread out in the lower range. C delay length based pruning effects on modularity. Position learning network achieves higher modularity. D delay length-based pruning effects on small-worldness. Position learning network achieves higher small-worldness.
• Figure 4: Architecture comparisons