Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A method for sparse and robust independent component analysis

Lauri Heinonen, Joni Virta

TL;DR

The paper introduces SICS, a semiparametric framework that achieves sparse and robust independent component analysis by coupling two robust scatter matrices with a sparsity-inducing penalty in a Li-style regression formulation. It provides a principled algorithm with an orthogonal-rotation correction, proves consistency for the single-component case, and characterizes robustness via breakdown points. Through simulations and real-data causal-discovery demonstrations, SICS shows improved performance under outliers and scalable behavior in higher dimensions, with a data-driven tool for selecting sparsity. The approach offers a flexible, interpretable ICA pipeline suitable for robust inference and causal graph construction in complex, high-dimensional settings.

Abstract

This work presents sparse invariant coordinate selection, SICS, a new method for sparse and robust independent component analysis. SICS is based on classical invariant coordinate selection, which is presented in such a form that a LASSO-type penalty can be applied to promote sparsity. Robustness is achieved by using robust scatter matrices. In the first part of the paper, the background and building blocks: scatter matrices, measures of robustness, ICS and independent component analysis, are carefully introduced. Then the proposed new method and its algorithm are derived and presented. This part also includes consistency and breakdown point results for a general case of sparse ICS-like methods. The performance of SICS in identifying sparse independent component loadings is investigated with multiple simulations. The method is illustrated with an example in constructing sparse causal graphs and we also propose a graphical tool for selecting the appropriate sparsity level in SICS.

A method for sparse and robust independent component analysis

TL;DR

The paper introduces SICS, a semiparametric framework that achieves sparse and robust independent component analysis by coupling two robust scatter matrices with a sparsity-inducing penalty in a Li-style regression formulation. It provides a principled algorithm with an orthogonal-rotation correction, proves consistency for the single-component case, and characterizes robustness via breakdown points. Through simulations and real-data causal-discovery demonstrations, SICS shows improved performance under outliers and scalable behavior in higher dimensions, with a data-driven tool for selecting sparsity. The approach offers a flexible, interpretable ICA pipeline suitable for robust inference and causal graph construction in complex, high-dimensional settings.

Abstract

This work presents sparse invariant coordinate selection, SICS, a new method for sparse and robust independent component analysis. SICS is based on classical invariant coordinate selection, which is presented in such a form that a LASSO-type penalty can be applied to promote sparsity. Robustness is achieved by using robust scatter matrices. In the first part of the paper, the background and building blocks: scatter matrices, measures of robustness, ICS and independent component analysis, are carefully introduced. Then the proposed new method and its algorithm are derived and presented. This part also includes consistency and breakdown point results for a general case of sparse ICS-like methods. The performance of SICS in identifying sparse independent component loadings is investigated with multiple simulations. The method is illustrated with an example in constructing sparse causal graphs and we also propose a graphical tool for selecting the appropriate sparsity level in SICS.

Paper Structure

This paper contains 22 sections, 17 theorems, 53 equations, 9 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Theoretical background
  3. ICA solution using two scatter matrices
  4. Regression formulation for scatter matrix ICA
  5. Symmetrized robust scatter matrices
  6. Measuring robustness
  7. Sparse and robust ICA
  8. Algorithm
  9. Consistency of SICS
  10. Robustness of SICS
  11. Computational aspects
  12. Simulations
  13. Simulation study #1: Benefits of robustness and sparsity
  14. Simulation study #2: Choosing the regularisation parameter
  15. Simulation study #3: Behavior under increasing $p$
  16. ...and 7 more sections

Key Result

Theorem 1

oja2006scatter Let $S$ be a scatter matrix and $x_1$, $x_2$ two independent copies of a random vector $x$. Then the symmetrized scatter matrix $S_s(x) := S(x_1-x_2)$ has the independence property.

Figures (9)

  • Figure 1: Average computation time of SICS as a function of sample size $n$ and data dimension $p$, divided into the computation of the scatters and the different parts of Algorithm \ref{['alg:SICS']}. The $y$-axis has a logarithmic scale.
  • Figure 2: Median absolute error by sample size for different methods. The error ribbon has width $0.2\times$MAD
  • Figure 3: Median absolute error by number of non-zero coefficients for different methods. The error ribbon has width $0.2\times$MAD. Error is scaled by dividing by $q$.
  • Figure 4: The plot shows the average optimal values of $r$ (as proportions of $p$) as a function of $\alpha$, the true proportion of non-zero coefficients in the first IC. The different lines correspond to different dimensions $p$.
  • Figure 5: The average estimation error as a function of $p$ in Simulation study #3
  • ...and 4 more figures

Theorems & Definitions (28)

  • Definition 1
  • Definition 2
  • Definition 3
  • Theorem 1
  • Definition 4
  • Theorem 2
  • Theorem 3
  • proof : Proof of Theorem \ref{['theo:ic_solution_2']}
  • Theorem 4
  • Theorem 5
  • ...and 18 more