Enhancing Small Object Encoding in Deep Neural Networks: Introducing Fast&Focused-Net with Volume-wise Dot Product Layer

TL;DR

This work tackles the challenge of efficiently encoding small objects into fixed-length vectors by introducing Fast&Focused-Net (FFN) and its Volume-wise Dot Product (VDP) layer. The VDP layer partitions the input feature map $H\times W\times C$ into non-overlapping hyper-volumes and applies per-volume dot products, achieving full patch coverage with reduced wasted computation and addressing CNN limitations in receptive field and early-layer bottlenecks. Empirically, FFN delivers state-of-the-art results on small-object benchmarks, and, when combined with Vision Transformers (ViT) variants such as ViT-C4 or multi-scale patching, achieves competitive or superior performance on large-image and text-recognition tasks (OpenImages V6, Places365, IC15, SVTP, etc.). These findings underscore FFN/VDP as a promising direction for efficient, focused deep representations, with potential extensions to video and hybrid CNN/Transformer architectures.

Abstract

In this paper, we introduce Fast&Focused-Net, a novel deep neural network architecture tailored for efficiently encoding small objects into fixed-length feature vectors. Contrary to conventional Convolutional Neural Networks (CNNs), Fast&Focused-Net employs a series of our newly proposed layer, the Volume-wise Dot Product (VDP) layer, designed to address several inherent limitations of CNNs. Specifically, CNNs often exhibit a smaller effective receptive field than their theoretical counterparts, limiting their vision span. Additionally, the initial layers in CNNs produce low-dimensional feature vectors, presenting a bottleneck for subsequent learning. Lastly, the computational overhead of CNNs, particularly in capturing diverse image regions by parameter sharing, is significantly high. The VDP layer, at the heart of Fast&Focused-Net, aims to remedy these issues by efficiently covering the entire image patch information with reduced computational demand. Experimental results demonstrate the prowess of Fast&Focused-Net in a variety of applications. For small object classification tasks, our network outperformed state-of-the-art methods on datasets such as CIFAR-10, CIFAR-100, STL-10, SVHN-Cropped, and Fashion-MNIST. In the context of larger image classification, when combined with a transformer encoder (ViT), Fast&Focused-Net produced competitive results for OpenImages V6, ImageNet-1K, and Places365 datasets. Moreover, the same combination showcased unparalleled performance in text recognition tasks across SVT, IC15, SVTP, and HOST datasets. This paper presents the architecture, the underlying motivation, and extensive empirical evidence suggesting that Fast&Focused-Net is a promising direction for efficient and focused deep learning.

Figures (4)

• Figure 1: Illustration highlighting the influence of various regions within the input feature map on the gradient determination for the weights of a convolutional layer's kernel.
• Figure 2: Comparative visualization of the Depthwise convolution layer and the VDP layer.
• Figure 3: Depiction of the Fast&Focused-Net backbone encoding network, assembled by stacking VDP Blocks. Each block consists of several VDP layers. For a detailed breakdown of each VDP layer, refer to Table \ref{['tab:volume_shapes']}.
• Figure 4: Visual illustration of Multi-Scale Patching.