Multi-Attribute Graph Estimation with Sparse-Group Non-Convex Penalties

TL;DR

This work tackles the problem of learning conditional independence graphs for high-dimensional multi-attribute Gaussian data by formulating a penalized log-likelihood with both elementwise and group sparsity. It analyzes convex sparse-group lasso and non-convex sparse-group log-sum and SCAD penalties, solved via an ADMM framework augmented with local linear approximation, and demonstrates local consistency, local convexity, and graph-recovery under two sets of conditions (with and without irrepresentability). Theoretical results are complemented by extensive numerical experiments on synthetic networks (ER and BA graphs) and real financial time-series data, showing that the log-sum penalty often outperforms lasso and SCAD in edge detection and recovery, while enabling meaningful multi-attribute graph structures. Overall, the paper extends multi-attribute graph learning to non-convex penalties, provides rigorous high-dimensional guarantees, and delivers practical improvements in graph estimation and interpretability for complex, vector-valued nodes.

Abstract

We consider the problem of inferring the conditional independence graph (CIG) of high-dimensional Gaussian vectors from multi-attribute data. Most existing methods for graph estimation are based on single-attribute models where one associates a scalar random variable with each node. In multi-attribute graphical models, each node represents a random vector. In this paper we provide a unified theoretical analysis of multi-attribute graph learning using a penalized log-likelihood objective function. We consider both convex (sparse-group lasso) and sparse-group non-convex (log-sum and smoothly clipped absolute deviation (SCAD) penalties) penalty/regularization functions. An alternating direction method of multipliers (ADMM) approach coupled with local linear approximation to non-convex penalties is presented for optimization of the objective function. For non-convex penalties, theoretical analysis establishing local consistency in support recovery, local convexity and precision matrix estimation in high-dimensional settings is provided under two sets of sufficient conditions: with and without some irrepresentability conditions. We illustrate our approaches using both synthetic and real-data numerical examples. In the synthetic data examples the sparse-group log-sum penalized objective function significantly outperformed the lasso penalized as well as SCAD penalized objective functions with $F_1$-score and Hamming distance as performance metrics.

Figures (1)

• Figure 1: Estimated precision matrices based edge weights for financial time series using proposed log-sum penalty with BIC for $\lambda$ selection. (a) Lasso, single-attribute case ($m=1$, $p=97$) with $|[\hat{\bm \Omega}]_{k \ell}|$, $k,\ell \in [97]$ as edge weight. (b) Lasso, multi-attribute case ($m=4$, $p=97$) with $\| \hat{\bm \Omega}^{(k \ell)} \|_F$, $k,\ell \in [97]$ as edge weight. (c) Log-sum, single-attribute case ($m=1$, $p=97$) with $|[\hat{\bm \Omega}]_{k \ell}|$, $k,\ell \in [97]$ as edge weight. (d) Log-sum, multi-attribute case ($m=4$, $p=97$) with $\| \hat{\bm \Omega}^{(k \ell)} \|_F$, $k,\ell \in [97]$ as edge weight. In all figures, the red squares (in dashed lines) show the 11 sectors -- they are not part of the edge weights.