Phase-Aware Deep Learning with Complex-Valued CNNs for Audio Signal Applications
Naman Agrawal
TL;DR
This work investigates Complex-Valued CNNs (CVCNNs) for audio signal processing, arguing that preserving phase information can yield richer representations beyond traditional magnitude-focused approaches. It builds a theoretical framework including complex convolutions, pooling, Wirtinger calculus, complex activations, complex batch normalization, and tailored weight initialization to enable stable learning in the complex domain. Empirically, CVCNNs achieve competitive results on image datasets and show potential gains in audio tasks, particularly when using phase-rich MFCC representations and cardioid activations; experiments with MFCCs and phase-aware graph structures reveal that phase can carry discriminative information, though fully leveraging it requires more sophisticated architectures. The findings suggest that complex representations hold practical value for audio processing and motivate future hybrid designs that combine phase-aware modules with CVCNNs to unlock deeper gains in real-world applications.
Abstract
This study explores the design and application of Complex-Valued Convolutional Neural Networks (CVCNNs) in audio signal processing, with a focus on preserving and utilizing phase information often neglected in real-valued networks. We begin by presenting the foundational theoretical concepts of CVCNNs, including complex convolutions, pooling layers, Wirtinger-based differentiation, and various complex-valued activation functions. These are complemented by critical adaptations of training techniques, including complex batch normalization and weight initialization schemes, to ensure stability in training dynamics. Empirical evaluations are conducted across three stages. First, CVCNNs are benchmarked on standard image datasets, where they demonstrate competitive performance with real-valued CNNs, even under synthetic complex perturbations. Although our focus is audio signal processing, we first evaluate CVCNNs on image datasets to establish baseline performance and validate training stability before applying them to audio tasks. In the second experiment, we focus on audio classification using Mel-Frequency Cepstral Coefficients (MFCCs). CVCNNs trained on real-valued MFCCs slightly outperform real CNNs, while preserving phase in input workflows highlights challenges in exploiting phase without architectural modifications. Finally, a third experiment introduces GNNs to model phase information via edge weighting, where the inclusion of phase yields measurable gains in both binary and multi-class genre classification. These results underscore the expressive capacity of complex-valued architectures and confirm phase as a meaningful and exploitable feature in audio processing applications. While current methods show promise, especially with activations like cardioid, future advances in phase-aware design will be essential to leverage the potential of complex representations in neural networks.