Statistical Analysis of the Impact of Quaternion Components in Convolutional Neural Networks

TL;DR

A statistical analysis carried out on experimental data is presented to compare the performance of existing components for the image classification problem and a novel Fully Quaternion ReLU activation function is introduced, which exploits the unique properties of quaternion algebra to improve model performance.

Abstract

In recent years, several models using Quaternion-Valued Convolutional Neural Networks (QCNNs) for different problems have been proposed. Although the definition of the quaternion convolution layer is the same, there are different adaptations of other atomic components to the quaternion domain, e.g., pooling layers, activation functions, fully connected layers, etc. However, the effect of selecting a specific type of these components and the way in which their interactions affect the performance of the model still unclear. Understanding the impact of these choices on model performance is vital for effectively utilizing QCNNs. This paper presents a statistical analysis carried out on experimental data to compare the performance of existing components for the image classification problem. In addition, we introduce a novel Fully Quaternion ReLU activation function, which exploits the unique properties of quaternion algebra to improve model performance.

Figures (6)

• Figure 1: Cause-and-effect diagram for the statistical design of experiments.
• Figure 2: Architecture used for classification of the MNIST dataset.
• Figure 3: Plot of group means with confidence intervals using data from the MNIST classification task. Left: Combination of initialization method, fully connected layer and number of parameters. Right: Combination of activation function, fully connected layer and number of parameters. The group with higher performance is selected in blue; the groups that are not statistically different from it are shown in gray, while the statistically different groups are shown in red.
• Figure 4: Architecture used for classification of the CIFAR-10 dataset.
• Figure 5: Plot of group means with confidence intervals using data from the CIFAR-10 classification task. The group with higher performance is selected in blue; the groups that are not statistically different from it are shown in gray, while the statistically different groups are shown in red.