Clusterpath Gaussian Graphical Modeling

TL;DR

The paper tackles the problem of high estimation variability in Gaussian Graphical Models by introducing CGGM, a convex clusterpath-based estimator that jointly learns a clustering of variables and the GGM parameters. By representing the precision matrix as Θ = U R U^T + A and penalizing inter-cluster differences with an aggregation term, CGGM induces a block-structured Θ whose inverse preserves the same block structure, while optionally incorporating a sparsity penalty. The authors develop a cyclic block coordinate descent algorithm with fusion steps and Newton updates, provide tuning and refitting procedures, and demonstrate strong performance in simulations across varied designs. They further show CGGM’s versatility through applications to finance, well-being indicators, and psychometrics, and discuss extending the framework to clustered covariance estimation. The work offers a practical, scalable approach to interpretable high-dimensional GGMs and provides software CGGMR for implementation.

Abstract

Graphical models serve as effective tools for visualizing conditional dependencies between variables. However, as the number of variables grows, interpretation becomes increasingly difficult, and estimation uncertainty increases due to the large number of parameters relative to the number of observations. To address these challenges, we introduce the Clusterpath estimator of the Gaussian Graphical Model (CGGM) that encourages variable clustering in the graphical model in a data-driven way. Through the use of an aggregation penalty, we group variables together, which in turn results in a block-structured precision matrix whose block structure remains preserved in the covariance matrix. The CGGM estimator is formulated as the solution to a convex optimization problem, making it easy to incorporate other popular penalization schemes which we illustrate through the combination of an aggregation and sparsity penalty. We present a computationally efficient implementation of the CGGM estimator by using a cyclic block coordinate descent algorithm. In simulations, we show that CGGM not only matches, but oftentimes outperforms other state-of-the-art methods for variable clustering in graphical models. We also demonstrate CGGM's practical advantages and versatility on a diverse collection of empirical applications.

Figures (16)

• Figure 1: Toy example of a graph representing the clustered precision matrix with $K=3$ clusters constructed from $p=8$ variables. Cluster variable $\xi_1$ is the average of the $p_1 = 4$ variables $X_1, X_2, X_3$, and $X_4$ having within-cluster conditional covariance $r_{11}$. Cluster variable $\xi_2$ is a singleton ($p_2 = 1$) and equal to the original variable $X_5$. Cluster variable $\xi_3$ is the average of the $p_3 = 3$ variables $X_6, X_7$, and $X_8$ having within-cluster conditional covariance $r_{33}$. The three cluster variables have conditional covariances $r_{12}, r_{13},$ and $r_{23}$.
• Figure 2: Precision matrices $\mathbf{\Theta}$ in the baseline simulation designs (top), the designs with clustering structure on the diagonal and with blockdiagonal structure, respectively, using balanced and unbalanced clusters sizes (bottom). The color shade indicates the magnitude of the elements. Diagonal elements are on a separate color scale than the off-diagonal ones to highlight their differing roles in CGGM in comparison to the benchmark methods.
• Figure 3: Results for the baseline simulation designs (columns). Top row: Boxplots of the Frobenius norm with black diamonds representing the average. Other rows: Diamonds displaying the average of the estimated number of clusters, ARI, FPR, and FNR. Reference lines are added for the true number of clusters, the ARI value of perfect clustering, and the FPR and FNR of perfect sparsity recognition, respectively. The size of the grey dots represents the frequency of different values across the replications. Aggregation performance is not applicable and omitted for GL and $\mathbf{S}^{-1}$, as is sparsity recognition performance for $\mathbf{S}^{-1}$. In the unstructured design, a misspecified tree for TAGL does not exist since any tree hierarchy contains the true clustering (each variable being its own cluster).
• Figure 4: Results for the simulation designs with clustering structure on the diagonal (left) and with a blockdiagonal structure (right). See Figure \ref{['fig:WB2022_baseline']} for explanatory notes. The FPR is not applicable in the design with clustering structure on the diagonal, since no elements of the true precision matrix are zero.
• Figure 5: Illustration of how an exact block structure is retained between the covariance matrix $\mathbf{\Sigma}$ and the precision matrix $\mathbf{\Theta}$ (left), whereas an approximate block structure in $\mathbf{\Sigma}$ corresponds to a much more noisy structure in $\mathbf{\Theta}$ (right).