NeuRN: Neuro-inspired Domain Generalization for Image Classification

TL;DR

This work tackles domain generalization in image classification by introducing NeuRN, a neuro-inspired patch-based normalization layer, and a Needleman-Wunsch–based framework for quantifying architectural similarity among DNNs. NeuRN is applied as a pre-processing layer to yield domain-agnostic feature representations and is evaluated across CNNs, Vision Transformers, and NAS-derived models on digit-domain transfers. Empirical results show consistent cross-domain gains across diverse architectures, with NAS models (SPOS and AutoFormer) also benefiting substantially from NeuRN. The study provides a dual contribution: a quantitative NW-based architectural similarity analysis and a practical neuro-inspired normalization strategy that enhances robustness to domain shifts, pointing toward broader neuro-inspired design principles for generalization.

Abstract

Domain generalization in image classification is a crucial challenge, with models often failing to generalize well across unseen datasets. We address this issue by introducing a neuro-inspired Neural Response Normalization (NeuRN) layer which draws inspiration from neurons in the mammalian visual cortex, which aims to enhance the performance of deep learning architectures on unseen target domains by training deep learning models on a source domain. The performance of these models is considered as a baseline and then compared against models integrated with NeuRN on image classification tasks. We perform experiments across a range of deep learning architectures, including ones derived from Neural Architecture Search and Vision Transformer. Additionally, in order to shortlist models for our experiment from amongst the vast range of deep neural networks available which have shown promising results, we also propose a novel method that uses the Needleman-Wunsch algorithm to compute similarity between deep learning architectures. Our results demonstrate the effectiveness of NeuRN by showing improvement against baseline in cross-domain image classification tasks. Our framework attempts to establish a foundation for future neuro-inspired deep learning models.

Figures (7)

• Figure 1: DNNs' architectural similarity: The confusion matrix on the left shows the correlation similarity of 44 DNN architectures, with darker clusters representing weaker correlation and lighter clusters representing stronger correlation. The panel on the right shows what the cells in the matrix represent: a 1:1 architectural comparison between artificial neural networks. The degree of similarity is calculated using the N-W algorithm, and every single layer is compared to build the similarity score.
• Figure 2: The dynamic matrix is constructed by filling in the cells iteratively using a set of recurrence relations between layers (denoted by characters). In evaluating each cell we make a choice by selecting the maximum of three possibilities - a match, a dissimilarity or a gap. Hence we determine the value of each (uninitialized) cell in the matrix by the cell to its left, above, and its diagonally left. A match and a dissimilarity are both represented as traveling in the diagonal direction as shown by the pointers. Finally, we traceback through the choices in the matrix from the bottom-right cell to the top-left cell which gives us the similarity index between two models. Left Matrix shows how similarity index between ResNet50 and VGG19 was calculated and right matrix shows a similar process for VGG19 and EfficientNetB0.
• Figure 3: Each x-label in the plot represents a layer combination and each marker represents the DNN which contains the corresponding combination of layers
• Figure 4: Each cell in this matrix represents the similarity index computed by number of common combinations across the entire model space. For instance, VGG19 contains 2xConv2D followed by a maxpool layer, any other models with this combination of layers in order would rank higher in similarity as compared to the ones that lack this combination.
• Figure S1: Based on Table 1, this matrix depicts the pairwise functional similarity of shortlisted deep neural network models resulting from the integration of NeuRN. The similarity is quantified using cosine similarity scores, derived from the models' performance on domain transfer tasks from MNIST datasets in Table 1. Positive cells represent increased functional similarity for NeuRN integrated models, Zero indicates no difference, while Negative indicates dissimilarity.