Wise-SrNet: A Novel Architecture for Enhancing Image Classification by Learning Spatial Resolution of Feature Maps

TL;DR

Wise-SrNet replaces Global Average Pooling with a depthwise-convolution–based mechanism that learns per-channel spatial processing of the final feature map, preserving spatial information without increasing computational cost. By applying a kernel that matches the feature-map's spatial extent and introducing non-negative constraints plus pre-averaging to mitigate overfitting, the approach achieves consistent improvements across 224×224 and 512×512 inputs on multiple backbones and datasets. The method demonstrates substantial gains in Top-1 accuracy (up to mid-singledigit to high-teens in some settings) and maintains similar parameter counts to GAP, addressing the reliability issues of GAP on large images and large-class problems. These results suggest Wise-SrNet as a robust, generalizable alternative to GAP for image classification and related tasks, with broad applicability and accessible code for replication.

Abstract

One of the main challenges since the advancement of convolutional neural networks is how to connect the extracted feature map to the final classification layer. VGG models used two sets of fully connected layers for the classification part of their architectures, which significantly increased the number of models' weights. ResNet and the next deep convolutional models used the Global Average Pooling (GAP) layer to compress the feature map and feed it to the classification layer. Although using the GAP layer reduces the computational cost, but also causes losing spatial resolution of the feature map, which results in decreasing learning efficiency. In this paper, we aim to tackle this problem by replacing the GAP layer with a new architecture called Wise-SrNet. It is inspired by the depthwise convolutional idea and is designed for processing spatial resolution while not increasing computational cost. We have evaluated our method using three different datasets: Intel Image Classification Challenge, MIT Indoors Scenes, and a part of the ImageNet dataset. We investigated the implementation of our architecture on several models of the Inception, ResNet, and DenseNet families. Applying our architecture has revealed a significant effect on increasing convergence speed and accuracy. Our Experiments on images with 224*224 resolution increased the Top-1 accuracy between 2% to 8% on different datasets and models. Running our models on 512*512 resolution images of the MIT Indoors Scenes dataset showed a notable result of improving the Top-1 accuracy within 3% to 26%. We will also demonstrate the GAP layer's disadvantage when the input images are large and the number of classes is not few. In this circumstance, our proposed architecture can do a great help in enhancing classification results. The code is shared at https://github.com/mr7495/image-classification-spatial.

Figures (9)

• Figure 1: The architectures of different classification layers with GAP, GWAP, and depthwise convolutional layer are portrayed in this figure.
• Figure 2: Classification with Flatten and Fully-Connected layers
• Figure 3: These figures represent the sex architecture we implemented for our experiments on images with 224×224 resolution. The red boxes are the model computational layers, and the blue boxes show the output arrays.
• Figure 4: This figure shows the validation accuracy of the trained models on the sub-ImageNet dataset. For each family, six versions of models (based on classification techniques) have been trained and evaluated to clarify our proposed architecture's performance. The images were resized to 224×224 pixels. The Averaging layer's kernel size was 2×2, and DP refers to a dropout layer with 50% neurons reduction.
• Figure 5: In this figure, the validation and training accuracies of the trained models on the Intel Image Classification dataset for 220 epochs are presented. The images were resized to 224×224 pixels. The Averaging layer's kernel size was 2×2.