Deep Deterministic Nonlinear ICA via Total Correlation Minimization with Matrix-Based Entropy Functional

TL;DR

DDICA tackles blind source separation under nonlinear mixing by directly minimizing total correlation using a matrix-based Rényi entropy estimator within a deep unmixing network. The method uses a single feed-forward autoencoder with a differentiable whitening layer and optimizes the TC objective via SGD, avoiding variational or adversarial surrogates. Empirical results across synthetic data, hyperspectral unmixing, modeling primary visual receptive fields, and resting-state fMRI show DDICA often outperforms linear ICA and competes with nonlinear ICA methods, with strong robustness to noise and complex mixtures. This approach provides a versatile, scalable nonlinear ICA framework with broad applicability in signal processing and neuroscience, offering biologically plausible representations and improved functional network discovery. The work also discusses practical limitations of matrix-based entropy and avenues for future enhancements, including joint spatial-time analysis and spatially constrained ICA.

Abstract

Blind source separation, particularly through independent component analysis (ICA), is widely utilized across various signal processing domains for disentangling underlying components from observed mixed signals, owing to its fully data-driven nature that minimizes reliance on prior assumptions. However, conventional ICA methods rely on an assumption of linear mixing, limiting their ability to capture complex nonlinear relationships and to maintain robustness in noisy environments. In this work, we present deep deterministic nonlinear independent component analysis (DDICA), a novel deep neural network-based framework designed to address these limitations. DDICA leverages a matrix-based entropy function to directly optimize the independence criterion via stochastic gradient descent, bypassing the need for variational approximations or adversarial schemes. This results in a streamlined training process and improved resilience to noise. We validated the effectiveness and generalizability of DDICA across a range of applications, including simulated signal mixtures, hyperspectral image unmixing, modeling of primary visual receptive fields, and resting-state functional magnetic resonance imaging (fMRI) data analysis. Experimental results demonstrate that DDICA effectively separates independent components with high accuracy across a range of applications. These findings suggest that DDICA offers a robust and versatile solution for blind source separation in diverse signal processing tasks.

Figures (9)

• Figure 1:
• Figure 2: Comparison of DDICA with 11 other linear ICA algorithms. Simulated ground-truth sources (sources 1, 2, and 3) and their corresponding noisy mixtures are shown for reference. To evaluate the performance of DDICA, we compared its results with those from 11 established linear ICA algorithms: Infomax, FastICA, Erica, Simbec, Evd, Jade Opac, Amuse, Radical ICA, Combi, ICA-EBM, and ERBM. The results highlight the ability of DDICA to recover sources with high accuracy and robustness compared to other linear ICA approaches.
• Figure 3: Quantitative evaluation of DDICA compared to 11 other linear ICA algorithms. To assess the performance of DDICA, we conducted 90 repeated simulation trials. In each trial, spatial correlation was measured between the estimated and ground truth components across three source signals. The average results for DDICA and 11 other linear ICA algorithms (Infomax, FastICA, Erica, Simbec, Evd, Jade Opac, Amuse, Radical ICA, Combi, ICA-EBM, and ERBM) are presented. This evaluation reveals that DDICA provides the highest performance for sources 1 and 2 and the highest overall performance across all methods and highlights the consistency and accuracy of DDICA in recovering independent sources across repeated experiments.
• Figure 4: Decomposing stronger nonlinear mixed sources. Strongly nonlinear signals such as blobs, spiral waves, and circular patterns were mixed to create extremely nonlinear mixed sources. The performance of top ICA algorithms, Infomax and FastICA, was then tested and compared against DDICA.
• Figure 5: Comparison of DDICA with other nonlinear ICA approaches. The ground truth sources and their corresponding mixture components were generated and visualized. Subsequently, four nonlinear ICA methods-iVAE, GIN, MISEP, and DDICA-were applied to the mixture components to perform source separation. The results of each method are presented separately for comparison. For DDICA, we additionally reported its performance with different stages of optimization.