Mice to Machines: Neural Representations from Visual Cortex for Domain Generalization

TL;DR

The paper addresses domain shift in vision by evaluating how mouse visual cortex representations align with deep learning feature representations. It introduces a generalized representational learning framework for population- and single-neuron–level comparisons and a neural response normalization (NeuRN) layer inspired by excitatory/inhibitory neuron dynamics. NeuRN improves both the alignment between artificial and biological representations and cross-domain generalization across MNIST, SVHN, USPS, and MNIST-M on multiple architectures, as evidenced by reductions in $RMSE$ and enhanced KDE/$IoU$ similarity. The work demonstrates that biology-inspired normalization yields more robust and biologically plausible AI systems with broad implications for neuroscience and artificial intelligence applications.

Abstract

The mouse is one of the most studied animal models in the field of systems neuroscience. Understanding the generalized patterns and decoding the neural representations that are evoked by the diverse range of natural scene stimuli in the mouse visual cortex is one of the key quests in computational vision. In recent years, significant parallels have been drawn between the primate visual cortex and hierarchical deep neural networks. However, their generalized efficacy in understanding mouse vision has been limited. In this study, we investigate the functional alignment between the mouse visual cortex and deep learning models for object classification tasks. We first introduce a generalized representational learning strategy that uncovers a striking resemblance between the functional mapping of the mouse visual cortex and high-performing deep learning models on both top-down (population-level) and bottom-up (single cell-level) scenarios. Next, this representational similarity across the two systems is further enhanced by the addition of Neural Response Normalization (NeuRN) layer, inspired by the activation profile of excitatory and inhibitory neurons in the visual cortex. To test the performance effect of NeuRN on real-world tasks, we integrate it into deep learning models and observe significant improvements in their robustness against data shifts in domain generalization tasks. Our work proposes a novel framework for comparing the functional architecture of the mouse visual cortex with deep learning models. Our findings carry broad implications for the development of advanced AI models that draw inspiration from the mouse visual cortex, suggesting that these models serve as valuable tools for studying the neural representations of the mouse visual cortex and, as a result, enhancing their performance on real-world tasks.

Figures (8)

• Figure 1: Block diagram architecture of comparison framework: image stimuli are presented to mice and DNN models (step 1) and their corresponding feature (step 2) and neural representations (step 3) are computed and systematically compared (step 4).
• Figure 2: A) RMSE score comparison of feature representations from NeuRN and non-NeuRN DNN models with MVC neural representations. Data points below the diagonal line signify better NeuRN models' biological plausibility with MVC neural representations. B) RMSE scores of feature representations' comparisons with neural representations across neural types for 13 (non-NeuRN) DNN models.
• Figure 3: Distributional comparison of NeuRN and non-NeuRN feature representations with neural representations is shown. KDE curves of randomly sampled excitatory and inhibitory neurons show NeuRN augmented feature representations are more similar to biological representations.
• Figure S1: A comprehensive overview of the Natural Scenes Dataset, which comprises a diverse array of real-world images. Each image presents a unique scene, contributing to the dataset's broad scope that spans across various landscapes, urban areas, and natural phenomena.
• Figure S3: A variety of DF/F traces representing activation of different neurons for various trials, providing insights into their unique signature of activity profile. Here x-axis is time in seconds and y-axis is neural amplitude.