Learning Discrete Latent Variable Structures with Tensor Rank Conditions

TL;DR

The paper addresses learning causal structure among unobserved discrete variables by establishing a tensor rank condition that ties the rank of observed joint contingency tensors to d-separation in the latent graph. It defines the Discrete Latent Structure Model (Discrete LSM) with measurement and structure submodels and key assumptions, then develops a two-stage identification algorithm: first locate latent variables via tensor-rank–driven causal clusters to identify the measurement model, then recover the latent-structure graph using a PC-style algorithm (PC-TENSOR-RANK) based on tensor-rank conditional independence tests. Practical procedures for estimating latent-dimension and testing tensor rank are provided via the CR statistic and a CP-decomposition-based goodness-of-fit test. Simulation studies and real-data analyses demonstrate improved latent-cluster recovery and structure discovery over baselines, confirming the method’s ability to identify non-tree, discrete latent-structure models and extend causal discovery with latent variables. This approach offers a principled, algebraic route to identifiability in discrete latent-variable causal discovery with pragmatic testing procedures.

Abstract

Unobserved discrete data are ubiquitous in many scientific disciplines, and how to learn the causal structure of these latent variables is crucial for uncovering data patterns. Most studies focus on the linear latent variable model or impose strict constraints on latent structures, which fail to address cases in discrete data involving non-linear relationships or complex latent structures. To achieve this, we explore a tensor rank condition on contingency tables for an observed variable set $\mathbf{X}_p$, showing that the rank is determined by the minimum support of a specific conditional set (not necessary in $\mathbf{X}_p$) that d-separates all variables in $\mathbf{X}_p$. By this, one can locate the latent variable through probing the rank on different observed variables set, and further identify the latent causal structure under some structure assumptions. We present the corresponding identification algorithm and conduct simulated experiments to verify the effectiveness of our method. In general, our results elegantly extend the identification boundary for causal discovery with discrete latent variables and expand the application scope of causal discovery with latent variables.

Figures (8)

• Figure 1: Illustrating for the graphical criteria of tensor rank condition such that a rank of the joint distribution is determined by the support of a specific conditional set that d-separates all observed variables, i.e., $\mathrm{Rank}(\mathbb{P}(X_1, X_2)) = |\mathrm{supp}(L)| = 2$.
• Figure 2: An example of discrete latent structure model involving 4 latent variables and 12 observed variables (sub-fig (a)). Here, the red edges form a measurement model, while the blue edges form a structural model. The theoretical result of this paper is shown in sub-fig (c).
• Figure 3: Example of the impure structure that can be identified by tensor rank condition.
• Figure 4: Illustrative example for $\mathcal{R}$ule 1.
• Figure 5: Illustrative example for $\mathcal{R}$ule 2.

Theorems & Definitions (42)

• Definition 2.1: Discrete Latent Structure Model
• Definition 3.1: Rank-one Tensor
• Definition 3.2: Tensor Rank kolda2009tensor
• Theorem 3.3: Graphical implication of tensor rank condition
• Example 3.4: Illustrating for the graphical criteria
• Definition 4.1: Causal cluster
• Proposition 4.2: Identification of support of latent variables
• Proposition 4.3: Identification of causal cluster
• Example 4.4: Finding causal clusters
• Proposition 4.5: Merging Rule