Blessing of Dependence: Identifiability and Geometry of Discrete Models with Multiple Binary Latent Variables

TL;DR

This work presents a general algebraic technique to investigate identifiability of complicated discrete models with latent and graphical components, and reveals an interesting and perhaps surprising phenomenon of blessing-of-dependence geometry.

Abstract

Identifiability of discrete statistical models with latent variables is known to be challenging to study, yet crucial to a model's interpretability and reliability. This work presents a general algebraic technique to investigate identifiability of discrete models with latent and graphical components. Specifically, motivated by diagnostic tests collecting multivariate categorical data, we focus on discrete models with multiple binary latent variables. We consider the BLESS model in which the latent variables can have arbitrary dependencies among themselves while the latent-to-observed measurement graph takes a "star-forest" shape. We establish necessary and sufficient graphical criteria for identifiability, and reveal an interesting and perhaps surprising geometry of blessing-of-dependence: under the minimal conditions for generic identifiability, the parameters are identifiable if and only if the latent variables are not statistically independent. Thanks to this theory, we can perform formal hypothesis tests of identifiability in the boundary case by testing marginal independence of the observed variables. In addition to the BLESS model, we also use the technique to show identifiability and the blessing-of-dependence geometry for a more flexible model, which has a general measurement graph beyond a start forest. Our results give new understanding of statistical properties of graphical models with latent variables. They also entail useful implications for designing diagnostic tests or surveys that measure binary latent traits.

Figures (4)

• Figure 1: Illustrating Proposition \ref{['prop-nece']}, severe consequence of lack of local identifiability. Left: the black line represents the true set of parameters and each colored line represents an alternative set of parameters. Right: marginal probability mass functions of the observed $\boldsymbol y \in \{0,1\}^5$ are plotted for all the parameter sets, "$+$" for the true set overlaid with circles "${\bigcirc}$" for 150 alternative sets.
• Figure 2: CPTs refer to Conditional Probability Tables. All nodes are discrete random variables, with $a_k\in\{0,1\}$ latent and $y_j\in\{1,\ldots,d\}$ observed. The parameters corresponding to the dashed directed edges in (b) are unidentifiable, because $a_1$ is indepedent of $\boldsymbol a_{2:5}$.
• Figure 3: Corroborating Theorem \ref{['thm-main']}. Two different views of the probability simplex $\mathcal{S}^3$ (tetrahedron) for the proportion parameters $\boldsymbol\nu$. The saddle surface $\mathcal{N}$ embedded in the simplex corresponds to the case with independent latent variables $a_1\perp\!\!\!\perp a_2$. Black dots correspond to the 20 parameter vectors $\boldsymbol\nu^{(m)}$ with the largest 20 MSEs among the 100 vectors $\boldsymbol\nu^{(1)},\ldots,\boldsymbol\nu^{(100)}\in \mathcal{S}^3$, and blue dots correspond to the remaining 80 parameter vectors.
• Figure 4: Two-latent-layer Bayesian Pyramid model in gu2023bp. Here the $\boldsymbol a$-layer-to-$\boldsymbol y$-layer measurement graph is a star tree, where each $a_k$ has exactly two children $y_{2k-1}$ and $y_{2k}$.

Theorems & Definitions (16)

• Definition 2.1: Strict Identifiability
• Definition 2.2: Generic Identifiability
• Definition 2.3: Local Identifiability
• Proposition 3.1: Necessary Condition for Generic Identifiability: $\geq 2$ children
• Theorem 3.2: Identifiability of the Latent-to-observed Star Forest $\mathbf G$
• Theorem 3.3: Blessing of Latent Dependence for the Two-children Case
• Proposition 3.4: Kruskal's Theorem Kicks in for the $\geq 3$ Children Case
• Corollary 3.5
• Lemma 3.6
• proof : Proof of Lemma \ref{['lem-poly']}