Learning Multi-Attribute Differential Graphs with Non-Convex Penalties

TL;DR

This work addresses estimating differences between two multi-attribute Gaussian graphical models by minimizing a penalized D-trace loss to recover the difference $\bm\Delta = \bm\Omega_y - \bm\Omega_x$ in high dimensions. It extends prior convex-group penalties to non-convex log-sum and SCAD penalties, proposing two proximal-gradient optimization schemes (LLA and nonconvexity redistribution) and providing theoretical guarantees (Theorems 1–2) for support recovery and convexity under appropriate conditions. The authors demonstrate that the log-sum penalty often yields sparser, more accurate differential graphs than lasso or SCAD in synthetic ER/BA graphs and in a real Beijing air-quality dataset, with model selection via a BIC-type criterion. Overall, the paper offers practical non-convex approaches with provable properties for reliable differential graph estimation in multi-attribute GGMs, enabling more precise characterization of conditional-dependency changes across conditions.

Abstract

We consider the problem of estimating differences in two multi-attribute Gaussian graphical models (GGMs) which are known to have similar structure, using a penalized D-trace loss function with non-convex penalties. The GGM structure is encoded in its precision (inverse covariance) matrix. Existing methods for multi-attribute differential graph estimation are based on a group lasso penalized loss function. In this paper, we consider a penalized D-trace loss function with non-convex (log-sum and smoothly clipped absolute deviation (SCAD)) penalties. Two proximal gradient descent methods are presented to optimize the objective function. Theoretical analysis establishing sufficient conditions for consistency in support recovery, convexity and estimation in high-dimensional settings is provided. We illustrate our approaches with numerical examples based on synthetic and real data.

Figures (1)

• Figure 1: Differential graphs comparing Beijing air-quality datasets Zhang2017 acquired from two sets of monitoring stations, 4 stations per set, year 2013-14: 4 monitoring stations and 11 features ($m=4$, $p=11$, $n_x=n_y=365$). Number of distinct edges $=35$ and $21$ in graphs (a) and (b), respectively. Estimated $\|\hat{\bm \Delta}^{(ij)}\|_F$ is the edge weight (normalized to have $\max_{i, j}\|\hat{\bm \Delta}^{(ij)}\|_F=1$). The edge weights are color coded , in addition to the edges with higher weights being drawn thicker.