Beyond independent component analysis: identifiability and algorithms

TL;DR

The paper addresses blind source separation under a relaxed independence assumption by introducing pairwise mean independence (PMI) and establishing identifiability results for PMICA, the PMI-variant of ICA. It develops an algebraic recovery approach based on higher-order cumulants, showing PMI induces a structured, orthogonally decomposable tensor representation that yields identifiability up to signed permutation when the sources are sufficiently general. By defining 'sufficiently general' cumulants and providing low-order, checkable conditions, the authors derive generic identifiability results and extend them to broader relaxations of independence. They then propose a minimum-distance estimator on the orthogonal group, prove consistency and asymptotic normality (with finite-sample sub-Gaussian bounds), and demonstrate via synthetic and EEG data that PMI-based methods can outperform traditional ICA when independence does not hold, with robust recovery of latent sources and artifacts. Overall, the work broadens the ICA framework to enable blind source separation under weaker, more realistic dependence structures while preserving rigorous identifiability and computable estimation guarantees.

Abstract

Independent Component Analysis (ICA) is a classical method for recovering latent variables with useful identifiability properties. For independent variables, cumulant tensors are diagonal; relaxing independence yields tensors whose zero structure generalizes diagonality. These models have been the subject of recent work in non-independent component analysis. We show that pairwise mean independence answers the question of how much one can relax independence: it is identifiable, any weaker notion is non-identifiable, and it contains the models previously studied as special cases. Our results apply to distributions with the required zero pattern at any cumulant tensor. We propose an algebraic recovery algorithm based on least-squares optimization over the orthogonal group. Simulations highlight robustness: enforcing full independence can harm estimation, while pairwise mean independence enables more stable recovery. These findings extend the classical ICA framework and provide a rigorous basis for blind source separation beyond independence.

Figures (8)

• Figure 1: Genericity conditions in the space of $d$-th order cumulant tensors of pairwise mean independent distributions. The gray plane is the this space, and the red loci are tensors with a non-unique orthogonal basis of eigenvectors. The pictures are three-dimensional slices explained in \ref{['remark:genericity-large-d']}.
• Figure 2: One million samples from $\mathbf{s}^{(\alpha)}$ for different $\alpha$. It is pairwise mean independent for all $\alpha$, because linear combinations of independent PMI vectors are PMI. It is independent when $\alpha = 0$ but not otherwise.
• Figure 3: RGD-PMICA outperforms ICA algorithms in recovering $\mathbf{s}^{(\alpha)}$. The ICA methods find the closest independent distribution, which is not the PMI one for $\alpha \geq 0.6$.
• Figure 4: Gap ($|\widehat{\kappa}_{1111} - \widehat{\kappa}_{2222}|$) and off-diagonal Frobenius norm for $\widehat{\kappa}_4(\mathbf{s}^{(\alpha)})$. ICA models recover the PMI sources when $\alpha$ is smaller than the ICA threshold.
• Figure 5: RGD-PMICA outperforms the ICA algorithms for all $n$. The decreasing trends of the ICA methods suggest that the closest independent distribution gets closer to the PMI one as $n$ increases.

Theorems & Definitions (54)

• Remark 1.1: Identifiability
• Definition 1.2
• Theorem 1.3: Identifiability of PMICA
• Theorem 1.4
• Theorem 1.5
• Theorem 2.1
• Remark 2.2
• Theorem 2.3
• proof
• Corollary 2.4