Nonparametric Evaluation of Noisy ICA Solutions

TL;DR

The paper tackles Blind Source Separation under Gaussian noise by introducing a nonparametric independence score based on the characteristic function, enabling data-driven selection among ICA algorithms without knowing noise parameters. It formalizes the independence score $\Delta(\mathbf{t},F|P)$, proves its consistency and uniform convergence, and presents two new CHF/CGF contrast functions that are robust to heavy tails and do not rely on higher moments. A convergence framework in pseudo-Euclidean space is developed, with global and local convergence guarantees for a broad class of contrasts, including the new CHF/CGF forms. Empirically, a Meta algorithm leveraging the independence score matches or surpasses the best candidate method across diverse distributions, noise levels, and tasks such as image demixing and MNIST denoising, highlighting practical impact for reliable ICA in noisy settings.

Abstract

Independent Component Analysis (ICA) was introduced in the 1980's as a model for Blind Source Separation (BSS), which refers to the process of recovering the sources underlying a mixture of signals, with little knowledge about the source signals or the mixing process. While there are many sophisticated algorithms for estimation, different methods have different shortcomings. In this paper, we develop a nonparametric score to adaptively pick the right algorithm for ICA with arbitrary Gaussian noise. The novelty of this score stems from the fact that it just assumes a finite second moment of the data and uses the characteristic function to evaluate the quality of the estimated mixing matrix without any knowledge of the parameters of the noise distribution. In addition, we propose some new contrast functions and algorithms that enjoy the same fast computability as existing algorithms like FASTICA and JADE but work in domains where the former may fail. While these also may have weaknesses, our proposed diagnostic, as shown by our simulations, can remedy them. Finally, we propose a theoretical framework to analyze the local and global convergence properties of our algorithms.

Figures (9)

• Figure 1: Meta-algorithm for choosing best candidate algorithm.
• Figure 2: Amari error in the $\log$-scale on $y$ axis and varying noise powers (for $n=10^5$) and varying sample sizes (for $\rho=0.2$) on $x$ axis for figures \ref{['exp:2']} and \ref{['exp:3']} respectively. For figure \ref{['exp:1']}, the top panel contains a histogram of 40 runs with one random initialization. The bottom panel contains a histogram of 40 runs, each of which is the best independence score out of 30 random initializations.
• Figure 3: We demix images using ICA by flattening and linearly mixing them with a $4 \times 4$ matrix $B$ (i.i.d entries $\sim \mathcal{N}(0,1)$) and Wishart noise ($\rho = 0.001$). The CHF-based method (c) recovers the original sources well, upto sign. The Kurtosis-based method (d) fails to recover the second source. This is consistent with its higher independence score. The Meta algorithm selects CHF from candidates CHF, CGF, Kurtosis, FastICA, and JADE. Appendix Section \ref{['section:ica_additional_experiments']} provides results for other contrast functions and their independence scores.
• Figure A.1: Image-Demixing using ICA
• Figure A.2: Image Denoising using ICA

Theorems & Definitions (33)

• Theorem 1
• Theorem 2
• Remark 1: Subgaussianity assumption
• Remark 2: Averaging over $\pmb{t}$
• Theorem 3
• Corollary 3.1
• proof
• Theorem 4
• Remark 3
• proof : Proof of Theorem \ref{['theorem:independence_score_correctness']}