Compressing Quaternion Convolutional Neural Networks for Audio Classification

TL;DR

The paper addresses the high computational cost of Quaternion Convolutional Neural Networks (QCNNs) for audio classification by proposing a compression pipeline that combines knowledge distillation (KD) and data-independent filter pruning. Empirical results show that pruning QCNNs typically matches or exceeds KD in preserving accuracy while reducing parameters and MACs, with substantial environmental benefits over uncompressed QCNNs. Across AudioSet and additional benchmarks (GTZAN, ESC-50, RAVDESS), pruned QCNNs achieve competitive performance with far fewer parameters and lower compute, highlighting practical deployment potential in resource-constrained settings. The work also provides a thorough analysis of generalization, environmental impact, and ablations, underscoring the value of quaternion architectures when coupled with efficient pruning.

Abstract

Conventional Convolutional Neural Networks (CNNs) in the real domain have been widely used for audio classification. However, their convolution operations process multi-channel inputs independently, limiting the ability to capture correlations among channels. This can lead to suboptimal feature learning, particularly for complex audio patterns such as multi-channel spectrogram representations. Quaternion Convolutional Neural Networks (QCNNs) address this limitation by employing quaternion algebra to jointly capture inter-channel dependencies, enabling more compact models with fewer learnable parameters while better exploiting the multi-dimensional nature of audio signals. However, QCNNs exhibit higher computational complexity due to the overhead of quaternion operations, resulting in increased inference latency and reduced efficiency compared to conventional CNNs, posing challenges for deployment on resource-constrained platforms. To address this challenge, this study explores knowledge distillation (KD) and pruning, to reduce the computational complexity of QCNNs while maintaining performance. Our experiments on audio classification reveal that pruning QCNNs achieves similar or superior performance compared to KD while requiring less computational effort. Compared to conventional CNNs and Transformer-based architectures, pruned QCNNs achieve competitive performance with a reduced learnable parameter count and computational complexity. On the AudioSet dataset, pruned QCNNs reduce computational cost by 50\% and parameter count by 80\%, while maintaining performance comparable to the conventional CNNs. Furthermore, pruned QCNNs generalize well across multiple audio classification benchmarks, including GTZAN for music genre recognition, ESC-50 for environmental sound classification and RAVDESS for speech emotion recognition.

Figures (14)

• Figure 1: Quaternion CNN compression pipeline:(a) Knowledge distillation and (b) Pruning with fine-tuning.
• Figure 2: Comparison of mAP across conventional and quaternion, unpruned and pruned CNNs.
• Figure 3: Pruned quaternion CNNs performance comparison with existing conventional CNNs and Transformer based models for Audio tagging.
• Figure 4: Convergence plots obtained across QCNN14 and QResNet38 for KD, Prune_FT and Prune_KD for audio tagging task. The convergence plot is obtained by averaging the convergence plots across different pruning ratios using the operator norm-based pruning method.
• Figure 5: Comparison of average fine-tuning time per iteration for pruned QCNN14 and QResNet38 models across different pruning ratios (0.25, 0.50, 0.75) using Prune_FT (pruning with fine-tuning) and Prune_KD (pruning with fine-tuning via KD) methods for audio tagging task. The models are trained on NVIDIA GPU (GeForce RTX 3090).