Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Non-Independent Components Analysis

Geert Mesters, Piotr Zwiernik

TL;DR

This paper documents new conditions that establish identification for several non-independent component models, e.g. common variance models, and proposes efficient estimation methods based on the identification results.

Abstract

A seminal result in the ICA literature states that for $AY = \varepsilon$, if the components of $\varepsilon$ are independent and at most one is Gaussian, then $A$ is identified up to sign and permutation of its rows (Comon, 1994). In this paper we study to which extent the independence assumption can be relaxed by replacing it with restrictions on higher order moment or cumulant tensors of $\varepsilon$. We document new conditions that establish identification for several non-independent component models, e.g. common variance models, and propose efficient estimation methods based on the identification results. We show that in situations where independence cannot be assumed the efficiency gains can be significant relative to methods that rely on independence.

Non-Independent Components Analysis

TL;DR

This paper documents new conditions that establish identification for several non-independent component models, e.g. common variance models, and proposes efficient estimation methods based on the identification results.

Abstract

A seminal result in the ICA literature states that for , if the components of are independent and at most one is Gaussian, then is identified up to sign and permutation of its rows (Comon, 1994). In this paper we study to which extent the independence assumption can be relaxed by replacing it with restrictions on higher order moment or cumulant tensors of . We document new conditions that establish identification for several non-independent component models, e.g. common variance models, and propose efficient estimation methods based on the identification results. We show that in situations where independence cannot be assumed the efficiency gains can be significant relative to methods that rely on independence.
Paper Structure (26 sections, 18 theorems, 107 equations, 6 figures, 2 tables)

This paper contains 26 sections, 18 theorems, 107 equations, 6 figures, 2 tables.

Table of Contents

  1. Omitted proofs -- main text
  2. Omitted proof from Section \ref{['sec:mainres']}
  3. Omitted proofs from Section \ref{['sec:inference']}
  4. Proof of Proposition \ref{['prop:assnormal']}
  5. Additional motivation
  6. Scaled Elliptical LiNGAM
  7. Invariance
  8. Alternative estimation methods
  9. Local identification beyond signed-permutations
  10. Moments and Cumulants --- some useful properties
  11. Combinatorial relationship between moments and cumulants
  12. Laws of total expectation and cumulance
  13. Estimating moments and cumulants
  14. k-statistics and sample cumulants
  15. Vectorizations of tensors
  16. ...and 11 more sections

Key Result

Theorem S1

Suppose that $\hat{\theta}$ minimizes $\hat{L}_n(\theta)$ over $\theta \in \Theta$. Assume that there exists a function $L_0(\theta)$ such that (a) $L_0(\theta)$ is uniquely minimized at $\theta_0$, (b) $L_0(\theta)$ is continuous, (c) $\Theta$ is compact and (d) $\sup_{\theta \in \Theta} | \hat{L}_

Figures (6)

  • Figure S1: Common Variance Experiments
  • Figure S2: Common Variance Experiments
  • Figure S3: Common Variance Experiments
  • Figure S4: Scaled Elliptical Experiments
  • Figure S5: Scaled Elliptical Experiments
  • ...and 1 more figures

Theorems & Definitions (35)

  • proof : Proof of Proposition \ref{['prop:binaryMI']}
  • proof : Proof of Lemma \ref{['lem:mainrephrase']}
  • proof : Proof of Proposition \ref{['prop:consist']}
  • Theorem S1
  • Lemma S2
  • proof
  • Theorem S3
  • Lemma S4
  • proof
  • Definition S1
  • ...and 25 more