TDSNNs: Competitive Topographic Deep Spiking Neural Networks for Visual Cortex Modeling

TL;DR

TDSNNs integrate deep spiking dynamics with a cortical-sheet topographic constraint via the Spatio-Temporal Constraints loss, enabling emergence of V1-to-IT topography in deep visual networks. The approach yields IT-like category patches, improved brain-likeness scores, and competitive ImageNet/CIFAR100 performance without accuracy loss, while also enhancing robustness to adversarial and noise perturbations. Key contributions include a novel dual-timescale STC objective, cortical-sheet neuron pre-optimization, and end-to-end training that preserves temporal processing alongside topography. The findings provide a biologically grounded framework for interpreting visual cortical organization and guiding the design of efficient, robust deep models with brain-inspired topology.

Abstract

The primate visual cortex exhibits topographic organization, where functionally similar neurons are spatially clustered, a structure widely believed to enhance neural processing efficiency. While prior works have demonstrated that conventional deep ANNs can develop topographic representations, these models largely neglect crucial temporal dynamics. This oversight often leads to significant performance degradation in tasks like object recognition and compromises their biological fidelity. To address this, we leverage spiking neural networks (SNNs), which inherently capture spike-based temporal dynamics and offer enhanced biological plausibility. We propose a novel Spatio-Temporal Constraints (STC) loss function for topographic deep spiking neural networks (TDSNNs), successfully replicating the hierarchical spatial functional organization observed in the primate visual cortex from low-level sensory input to high-level abstract representations. Our results show that STC effectively generates representative topographic features across simulated visual cortical areas. While introducing topography typically leads to significant performance degradation in ANNs, our spiking architecture exhibits a remarkably small performance drop (No drop in ImageNet top-1 accuracy, compared to a 3% drop observed in TopoNet, which is the best-performing topographic ANN so far) and outperforms topographic ANNs in brain-likeness. We also reveal that topographic organization facilitates efficient and stable temporal information processing via the spike mechanism in TDSNNs, contributing to model robustness. These findings suggest that TDSNNs offer a compelling balance between computational performance and brain-like features, providing not only a framework for interpreting neural science phenomena but also novel insights for designing more efficient and robust deep learning models.

Figures (17)

• Figure 1: TDSNNs leverage temporal information. (Left) Spike train entropy shifts reveal topography-dependent temporal dynamics across various inference timesteps. (Entropy is derived from the neurons' firing probabilities). (Right) TDSNNs' spiking mechanisms inherently solve topographic ANNs' persistent recognition degradation by leveraging these temporal patterns.
• Figure 2: Overview of the methodology for inducing visual cortex-like neural organization in SNN architectures.(a) Illustration of the virtual 2D cortical sheet assigned to each layer of the SNN. Each scatter represents a neuron. (b) Spatio-Temporal Constraints (STC) is designed to promote similar response patterns in spatially nearby neurons across both long-time and short-time scales. (c) Schematic of the Training Pipeline for TDSNNs.
• Figure 3: Analysis of V1-like topography of TDSNNs.(a) Sine grating stimuli used to probe neural responses, as described in margalit2024unifying. (b) Preference maps for orientation, spatial frequency, and color in Layer 2.0. Top row: non-topographic SResNet18. Bottom row: topographic TSResNet18. Orientation preference maps were generated via vector summation of angle-specific response data bosking1997orientation. (c) Smoothness analysis of orientation, spatial frequency, and color preferences. Higher smoothness denote greater similarity in responses among closely located neurons, indicating smoother transitions in preference maps. (See Appendix for details; error bars: SEM). (d) Pairwise firing rate correlation as a function of spatial distance for orientation preference (with 95% confidence intervals).
• Figure 4: Analysis of IT-like topography of TDSNNs.(a) Category t-value selectivity maps of the final layer are shown for SResNet18 (non-topo) and TSResNet18 (topo). The topographic organization in TSResNet18 exhibits a more clustered "continent" form, indicating larger neural clusters for similar functional representations. Areas of high selectivity for faces overlap with those for bodies, whereas areas for characters and places are spatially segregated, as indicated by black dots. (See Appendix for t-value definition.) (b) Difference in selectivity as a function of pairwise neuronal distance for bodies and objects. (c) Smoothness analysis of faces and bodies t-value maps.
• Figure 5: Overlap correlation of selectivity maps calculated for the four fLoc stimulus classes. (See Appendix for how we identify the significant selectivity patches.)