High-Dimensional Undirected Graphical Models for Arbitrary Mixed Data

TL;DR

This work develops a scalable framework for learning high-dimensional graphical models from general mixed data by embedding them in a latent Gaussian copula (LGCM). It unifies three cases—continuous–continuous (Case I), ordinal–continuous (Case II), and ordinal–ordinal (Case III)—through estimators for the latent correlation matrix $\boldsymbol{\Sigma}$ based on polychoric/polyserial ideas and threshold estimation, followed by sparse precision matrix recovery via graphical lasso with data-driven model selection. The authors establish concentration results and convergence rates for the estimators in both latent Gaussian and nonparanormal settings, show favorable performance in extensive simulations against bridge-function methods, and demonstrate applicability to UK Biobank COVID-19 risk-factor data. They provide practical software (R package hume) and reproducible code, enabling researchers to apply these methods to diverse mixed-type datasets. Overall, the approach offers a principled, flexible, and scalable path to robust mixed-data graphical modeling in high dimensions.

Abstract

Graphical models are an important tool in exploring relationships between variables in complex, multivariate data. Methods for learning such graphical models are well developed in the case where all variables are either continuous or discrete, including in high-dimensions. However, in many applications data span variables of different types (e.g. continuous, count, binary, ordinal, etc.), whose principled joint analysis is nontrivial. Latent Gaussian copula models, in which all variables are modeled as transformations of underlying jointly Gaussian variables, represent a useful approach. Recent advances have shown how the binary-continuous case can be tackled, but the general mixed variable type regime remains challenging. In this work, we make the simple yet useful observation that classical ideas concerning polychoric and polyserial correlations can be leveraged in a latent Gaussian copula framework. Building on this observation we propose flexible and scalable methodology for data with variables of entirely general mixed type. We study the key properties of the approaches theoretically and empirically, via extensive simulations as well an illustrative application to data from the UK Biobank concerning COVID-19 risk factors.

Figures (8)

• Figure 1: Simulation results for the binary-continuous data setting based on $100$ simulation runs. The left column corresponds to the latent Gaussian model, where the transformation function is the identity. The right colum depicts results for the LGCM with $f_j(x) = x^{1/3}$ for all $j$. The top row reports mean and standard deviation of the AUC along simulation runs, and the bottom row depicts boxplots of the estimation error $\Norm{\hat{\mathbf\Omega} - \mathbf\Omega^*}_F$. The y-axis labels have superscript arrows attached to indicate the direction of improvement: $\rightarrow$ implies that larger values are better, and $\leftarrow$ implies that smaller values are better.
• Figure 2: Simulation results for the general mixed data setting based on $100$ simulation runs. The left column corresponds to the latent Gaussian model, where the transformation function is the identity. The right colum depicts results for the LGCM with $f_j(x) = x^{1/3}$ for all $j$. The top row reports mean and standard deviation of the AUC along simulation runs, and the bottom row depicts boxplots of the estimation error $\Norm{\hat{\mathbf\Omega} - \mathbf\Omega^*}_F$. The y-axis labels have superscript arrows attached to indicate the direction of improvement: $\rightarrow$ implies that larger values are better, and $\leftarrow$ implies that smaller values are better.
• Figure 3: Comparison of the Case II MLE under the LGCM and the ad hoc estimator.
• Figure 4: Computation time in milliseconds for the Case II MLE and ad hoc estimators. We report the median (solid line) and the first and third quartile (shaded area) of recorded computation time. In the left panel, we compare computation time against a grid of sample sizes $n \in [50, 10000]$ with a step size of $s_t = 50$. In the right panel, we compare computation time against a grid of true correlation values $\Sigma^*_{jk} \in [-.98, .98]$.
• Figure 5: Simulation results for the binary-continuous data setting based on $100$ simulation runs. The left column corresponds to the latent Gaussian model, where the transformation function is the identity. The right colum depicts results for the LGCM with $f_j(x) = x^{1/3}$ for all $j$. The top and bottom rows report mean and standard deviation of the TPR and FPR along simulation runs, respectively. The y-axis labels have superscript arrows attached to indicate the direction of improvement: $\rightarrow$ implies that larger values are better.

Theorems & Definitions (24)

• Definition 2.1: The nonparanormal model
• Definition 2.2: latent Gaussian copula model for general mixed data
• Definition 2.3: MLE $\hat{\mathbf{\Sigma}}^{(n)}$ of $\mathbf{\Sigma}$; Case I
• Definition 2.4: MLE $\hat{\mathbf{\Sigma}}^{(n)}$ of $\mathbf{\Sigma}$; Case II
• Definition 2.5: MLE $\hat{\mathbf{\Sigma}}^{(n)}$ of $\mathbf{\Sigma}$; Case III
• Definition 3.1: Nonparanormal estimator $\hat{\mathbf{\Sigma}}^{(n)}$ of $\mathbf{\Sigma}$; Case I
• Definition 3.2: Estimator $\hat{\mathbf{\Sigma}}^{(n)}$ of $\mathbf{\Sigma}$; Case II nonparanormal
• Definition 3.3: Nonparanormal estimator $\hat{\mathbf{\Sigma}}^{(n)}$ of $\mathbf{\Sigma}$; Case III
• Lemma 3.1
• Theorem 3.2