Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convolutional Neural Networks Do Work with Pre-Defined Filters

Christoph Linse, Erhardt Barth, Thomas Martinetz

TL;DR

A novel class of Convolutional Neural Networks called Pre-defined Filter ConvolutionAL Neural Networks (PFCNNs), where all convolution kernels with n > 1 are pre-defined and constant during training, which provides a novel perspective on the way information is processed within deep CNNs.

Abstract

We present a novel class of Convolutional Neural Networks called Pre-defined Filter Convolutional Neural Networks (PFCNNs), where all nxn convolution kernels with n>1 are pre-defined and constant during training. It involves a special form of depthwise convolution operation called a Pre-defined Filter Module (PFM). In the channel-wise convolution part, the 1xnxn kernels are drawn from a fixed pool of only a few (16) different pre-defined kernels. In the 1x1 convolution part linear combinations of the pre-defined filter outputs are learned. Despite this harsh restriction, complex and discriminative features are learned. These findings provide a novel perspective on the way how information is processed within deep CNNs. We discuss various properties of PFCNNs and prove their effectiveness using the popular datasets Caltech101, CIFAR10, CUB-200-2011, FGVC-Aircraft, Flowers102, and Stanford Cars. Our implementation of PFCNNs is provided on Github https://github.com/Criscraft/PredefinedFilterNetworks

Convolutional Neural Networks Do Work with Pre-Defined Filters

TL;DR

A novel class of Convolutional Neural Networks called Pre-defined Filter ConvolutionAL Neural Networks (PFCNNs), where all convolution kernels with n > 1 are pre-defined and constant during training, which provides a novel perspective on the way information is processed within deep CNNs.

Abstract

We present a novel class of Convolutional Neural Networks called Pre-defined Filter Convolutional Neural Networks (PFCNNs), where all nxn convolution kernels with n>1 are pre-defined and constant during training. It involves a special form of depthwise convolution operation called a Pre-defined Filter Module (PFM). In the channel-wise convolution part, the 1xnxn kernels are drawn from a fixed pool of only a few (16) different pre-defined kernels. In the 1x1 convolution part linear combinations of the pre-defined filter outputs are learned. Despite this harsh restriction, complex and discriminative features are learned. These findings provide a novel perspective on the way how information is processed within deep CNNs. We discuss various properties of PFCNNs and prove their effectiveness using the popular datasets Caltech101, CIFAR10, CUB-200-2011, FGVC-Aircraft, Flowers102, and Stanford Cars. Our implementation of PFCNNs is provided on Github https://github.com/Criscraft/PredefinedFilterNetworks

Paper Structure

This paper contains 18 sections, 1 equation, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Pre-defined Filter Networks
  4. Pre-defined Filter Module
  5. Choice of the parameter f
  6. PFNet18 architecture
  7. Choice of pre-defined filters
  8. Experimental setup
  9. Datasets
  10. Training details
  11. Results
  12. Benchmarks
  13. Feature visualization
  14. Computational efficiency
  15. Aliasing effects
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Pre-defined Filter Module (PFM). $1 \times n \times n$ kernels are taken from a small pool of kernels and are applied channel-wise as known from depthwise convolution. The order in which the kernels are distributed over the input channels is fixed.
  • Figure 2: 8 uneven and 8 even $1 \times 3 \times 3$ convolution kernels used in PFNet18.
  • Figure 3: Aliasing effects. Various stride 2 convolutional operations are applied on the input image showing two horizontal lines. Orange color denotes positive kernel weights, black denotes zero, and blue negative weights.