Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reservoir Subspace Injection for Online ICA under Top-n Whitening

Wenjun Xiao, Yuda Bi, Vince D Calhoun

TL;DR

Reservoir expansion can improve online independent component analysis (ICA) under nonlinear mixing, yet top-$n$ whitening may discard injected features, which formalizes this bottleneck as RSI (RSI): injected features help only if they enter the retained eigenspace without displacing passthrough directions.

Abstract

Reservoir expansion can improve online independent component analysis (ICA) under nonlinear mixing, yet top-$n$ whitening may discard injected features. We formalize this bottleneck as \emph{reservoir subspace injection} (RSI): injected features help only if they enter the retained eigenspace without displacing passthrough directions. RSI diagnostics (IER, SSO, $ρ_x$) identify a failure mode in our top-$n$ setting: stronger injection increases IER but crowds out passthrough energy ($ρ_x: 1.00\!\rightarrow\!0.77$), degrading SI-SDR by up to $2.2$\,dB. A guarded RSI controller preserves passthrough retention and recovers mean performance to within $0.1$\,dB of baseline $1/N$ scaling. With passthrough preserved, RE-OICA improves over vanilla online ICA by $+1.7$\,dB under nonlinear mixing and achieves positive SI-SDR$_{\mathrm{sc}}$ on the tested super-Gaussian benchmark ($+0.6$\,dB).

Reservoir Subspace Injection for Online ICA under Top-n Whitening

TL;DR

Reservoir expansion can improve online independent component analysis (ICA) under nonlinear mixing, yet top- whitening may discard injected features, which formalizes this bottleneck as RSI (RSI): injected features help only if they enter the retained eigenspace without displacing passthrough directions.

Abstract

Reservoir expansion can improve online independent component analysis (ICA) under nonlinear mixing, yet top- whitening may discard injected features. We formalize this bottleneck as \emph{reservoir subspace injection} (RSI): injected features help only if they enter the retained eigenspace without displacing passthrough directions. RSI diagnostics (IER, SSO, ) identify a failure mode in our top- setting: stronger injection increases IER but crowds out passthrough energy (), degrading SI-SDR by up to \,dB. A guarded RSI controller preserves passthrough retention and recovers mean performance to within \,dB of baseline scaling. With passthrough preserved, RE-OICA improves over vanilla online ICA by \,dB under nonlinear mixing and achieves positive SI-SDR on the tested super-Gaussian benchmark (\,dB).
Paper Structure (9 sections, 3 theorems, 14 equations, 2 figures, 2 tables)

This paper contains 9 sections, 3 theorems, 14 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Model and Main Results
  3. Setup and Notation
  4. Reservoir Encoding
  5. Motivation: Random-Feature Approximation
  6. Reservoir Subspace Injection (RSI)
  7. Online Whitening and Natural Gradient ICA
  8. Experiments
  9. Conclusion

Key Result

Proposition 1

Let $\sigma$ be $L_\sigma$-Lipschitz. If $\rho_{\mathrm{eff}}:=(1-\alpha_r)+\alpha_r L_\sigma\|\mathbf{W}_{\mathrm{res}}\|<1$, then for any two initial states $\mathbf{r}_0,\mathbf{r}_0'$ driven by the same input, $\|\mathbf{r}_t-\mathbf{r}_t'\|\le\rho_{\mathrm{eff}}^{\,t}\|\mathbf{r}_0-\mathbf{r}_0

Figures (2)

  • Figure 1: RE-OICA validation ($T{=}15{,}000$, $n{=}3$, 10 seeds). (a) Running unshifted SI-SDR (trailing 2 000): RE-OICA, vanilla, FastICA. (b) Steady-state SI-SDR$_{\mathrm{sc}}$ across regimes (lag-compensated). (c) Nonlinear overlay (last 600): true vs. RE-OICA; corner labels show per-source unshifted SI-SDR.
  • Figure 2: Ablation studies (10 seeds, mean$\pm$SEM). (a)$N$-sweep under nonlinear mixing across three scaling branches. (b) Architecture: ESN vs. RF vs. vanilla (time-varying). (c) Drift sweep: $\varepsilon\in\{0.1,0.3,0.8\}$.

Theorems & Definitions (5)

  • Proposition 1: Echo State Property
  • proof
  • Proposition 2: Motivational linearization bound
  • Proposition 3: Reservoir-entry condition (block-diagonal case)
  • proof : Sketch