Explicitly Modeling Subcortical Vision with a Neuro-Inspired Front-End Improves CNN Robustness

TL;DR

CNN robustness gaps relative to biological vision are addressed by EVNets, which explicitly model subcortical vision through a fixed SubcorticalBlock paired with a VOneBlock front end before a CNN backend. The front-end cascade implements a center–surround DoG with light adaptation and contrast normalization, plus a noise generator, forming a fixed, neuro-inspired early-vision pipeline with a $7^ ^ ightarrow$FoV and extended spatial-frequency coverage via the Gabor filter bank. EVNets achieve stronger V1 alignment (including extra-classical receptive-field properties) and a substantial 9.3% gain in a robustness score over the baseline CNN, with additional additive gains when combined with PRIME data augmentation. The improvements generalize across back-ends and arise from complementary architectural priors and training-based strategies, suggesting a practical path to more robust, brain-aligned vision systems.

Abstract

Convolutional neural networks (CNNs) trained on object recognition achieve high task performance but continue to exhibit vulnerability under a range of visual perturbations and out-of-domain images, when compared with biological vision. Prior work has demonstrated that coupling a standard CNN with a front-end (VOneBlock) that mimics the primate primary visual cortex (V1) can improve overall model robustness. Expanding on this, we introduce Early Vision Networks (EVNets), a new class of hybrid CNNs that combine the VOneBlock with a novel SubcorticalBlock, whose architecture draws from computational models in neuroscience and is parameterized to maximize alignment with subcortical responses reported across multiple experimental studies. Without being optimized to do so, the assembly of the SubcorticalBlock with the VOneBlock improved V1 alignment across most standard V1 benchmarks, and better modeled extra-classical receptive field phenomena. In addition, EVNets exhibit stronger emergent shape bias and outperform the base CNN architecture by 9.3% on an aggregate benchmark of robustness evaluations, including adversarial perturbations, common corruptions, and domain shifts. Finally, we show that EVNets can be further improved when paired with a state-of-the-art data augmentation technique, surpassing the performance of the isolated data augmentation approach by 6.2% on our robustness benchmark. This result reveals complementary benefits between changes in architecture to better mimic biology and training-based machine learning approaches.

Figures (8)

• Figure 1: Simulating primate early visual processing as CNN front-end blocks.A The SubcorticalBlock integrates two parallel processing pathways for P and M cells with a light-adaptation layer, a DoG convolutional layer, a contrast-normalization layer and a neural noise generator. B Both VOneNets and EVNets comprise an initial block designed to simulate a specific stage of the visual system, followed by a standard CNN architecture. VOneNets include a VOneBlock and EVNets include both a SubcorticalBlock and a VOneBlock. C SF, size, and contrast tuning curves (left to right) for two example subcortical neurons with example frames from the drifting gratings stimulus set shown below. Markers indicate the F1 component of the cell response and the solid line depicts the fitted response functions used for parameterizing response properties (cf. Supplementary Material \ref{['sec:sup_subcorticalblock_implementation']}). Notably, the SubcorticalBlock exhibits hallmark LGN phenomena, including contrast saturation and surround suppression, with stronger modulation observed in M cells.
• Figure 2: Subcortical preprocessing improves explanability of extra-classical RF properties in V1. SF, size, and contrast tuning curves (left to right) for an example neuron in the VOneBlock with and without subcortical preprocessing (M cell). Example frames from the drifting gratings stimuli are shown below. VOneBlock neurons in isolation exhibit predominantly classical RF effects but when coupled with subcortical processing exhibit behaviors consistent with those empirically observed, such as enhanced surround modulation and non-linear contrast responsesdoi:10.1152/jn.00692.2001SCLAR19901. See Supplementary Material \ref{['sec:empirical_tuning_curves']} for empirical V1 tuning curves.
• Figure B1: Examples of empirical V1 tuning curves retrieved from the literature. SF, size, and contrast tuning curves (left to right) for example V1 neurons. Left SF tuning curve of a simple (gray) and complex (black) cell to drifting grating stimuli. Markers represent the total number of F1 responses to gratings of different SF normalized to the best response and the solid line depicts a quadratic fit for purposes of illustrating the tuning profile (adapted from Figure 10 in Schiller et al. doi:10.1152/jn.1976.39.6.1334). Middle Size tuning curve of two complex cells of V1 with distinct degrees of surround suppression under high contrasts. Gray depicts the a cell form V1 layer 4B under 0.15 contrast and black represents a cell from V1 layer 6 under 0.31 contrast. Markers represent each cell's F1 response to differently-sized gratings and the line depicts the predicted response of a fitted DoG model discussed in the original article (adapted from Figure 1 in Sceniak et al. Sceniak1999). Right Contrast tuning curve of two simple V1 cell from the least (gray) and most (black) contrast sensitive thirds of their respective population. Marks indicate F1 response and the solid line depicts a fitted response model discussed in the original article (adapted from Figure 2 in Sclar et al. SCLAR19901). Data points extracted via WebPlotDigitalizer WebPlotDigitizer.
• Figure C1: Adversarial robustness is evaluated at convergance of PGD iterations. Top-1 white-box accuracy iteration curves for PGD attacks with $\|\delta\|_\infty=1/255$, $\|\delta\|_2=0.6$, $\|\delta\|_1=160$ constraints for ResNet50, VOneResNet50 and EVResNet50 models, evaluated on 500 images. The step size was adjusted to be $\epsilon$ for 1 iterations, and $2\epsilon/N$, in the remaining cases. Increasing the number of PGD iteration steps increases attack effectiveness only up to roughly 32 iterations. Lines indicate the mean accuracy and shaded error bars denote SD ($n=3$ seeds).
• Figure C2: Examples of common image corruptions from the ImageNet-C dataset at intermediate severity (level 3) The first row shows the original image and three noise corruptions; the second row displays blur corruptions; the third row presents weather-related corruptions; and the fourth row illustrates digital corruptions.