Nonparametric Factor Analysis and Beyond

TL;DR

The paper addresses latent-variable identifiability in highly general nonparametric, nonlinear, and noisy settings by leveraging a Hu–Schennach-inspired framework. It proves distribution identifiability and, under structural or distributional variability, component-wise identifiability of latent factors, even when the generating function is noninvertible and noise is non-negligible. To operationalize these insights, it introduces two estimation approaches: GEEN, a KL-divergence-based method with kernel density estimates for univariate latents, and a Regularized Autoencoder (RAE) that enforces conditional independence and likelihood-based learning for multivariate latents. Empirical validation includes simulations across continuous and discrete latents and a real-world GDP refinement experiment showing that latent GDP estimates can reveal deeper economic patterns than official measures. Collectively, the work provides a principled path from general identifiability theory to practical latent-variable estimation and meaningful real-world applications.

Abstract

Nearly all identifiability results in unsupervised representation learning inspired by, e.g., independent component analysis, factor analysis, and causal representation learning, rely on assumptions of additive independent noise or noiseless regimes. In contrast, we study the more general case where noise can take arbitrary forms, depend on latent variables, and be non-invertibly entangled within a nonlinear function. We propose a general framework for identifying latent variables in the nonparametric noisy settings. We first show that, under suitable conditions, the generative model is identifiable up to certain submanifold indeterminacies even in the presence of non-negligible noise. Furthermore, under the structural or distributional variability conditions, we prove that latent variables of the general nonlinear models are identifiable up to trivial indeterminacies. Based on the proposed theoretical framework, we have also developed corresponding estimation methods and validated them in various synthetic and real-world settings. Interestingly, our estimate of the true GDP growth from alternative measurements suggests more insightful information on the economies than official reports. We expect our framework to provide new insight into how both researchers and practitioners deal with latent variables in real-world scenarios.

Figures (5)

• Figure 1: Results w.r.t. different numbers of latent variables.
• Figure 2: Results w.r.t. different standard deviations ($\sigma$) of the noise.
• Figure 3: Country examples of official and refined GDP growth.
• Figure 4: Results w.r.t. different numbers of latent variables for model satisfying distributional variability.
• Figure 5: Results w.r.t. different standard deviations ($\sigma$) of the noise for model satisfying distributional variability. We set the number of latent variables $n$ as $5$.

Theorems & Definitions (16)

• Theorem 1
• Theorem 2
• Lemma 1
• Lemma 2
• Theorem 3
• Theorem 4
• Theorem 4
• proof
• Lemma 2
• proof