Generating Human Understandable Explanations for Node Embeddings

TL;DR

This work tackles the opacity of node embeddings by introducing XM, a framework that augments existing embedding methods with human-understandable sense features. It defines an Explain matrix $E \in \mathbb{R}^{d\times f}$ that links embedding dimensions to features and minimizes the nuclear norm $||E||_*$ via sparsity and orthogonality constraints, producing denoised, interpretable explanations without sacrificing performance. XM is validated across multiple algorithms and real-world graphs, showing reduced Explain-norms and comparable AUC in link prediction, with ablation indicating the orthogonality constraint as a key driver of explainability. The approach offers a task-agnostic, extensible path to interpretable embeddings by enabling per-dimension explanations linked to structural graph properties, suitable for cross-domain graphs and large-scale networks.

Abstract

Node embedding algorithms produce low-dimensional latent representations of nodes in a graph. These embeddings are often used for downstream tasks, such as node classification and link prediction. In this paper, we investigate the following two questions: (Q1) Can we explain each embedding dimension with human-understandable graph features (e.g. degree, clustering coefficient and PageRank). (Q2) How can we modify existing node embedding algorithms to produce embeddings that can be easily explained by human-understandable graph features? We find that the answer to Q1 is yes and introduce a new framework called XM (short for eXplain eMbedding) to answer Q2. A key aspect of XM involves minimizing the nuclear norm of the generated explanations. We show that by minimizing the nuclear norm, we minimize the lower bound on the entropy of the generated explanations. We test XM on a variety of real-world graphs and show that XM not only preserves the performance of existing node embedding methods, but also enhances their explainability.

Figures (16)

• Figure 1: We show the sense making process using a simple toy example of a barbell graph. We embed the graph into 16 dimensions using DGI and show the Explain matrices for a bridge node (shown in pink) and a random node in a clique (shown in grey). The columns ($x$ axis) of the Explain matrix correspond to the sense features and the rows ($y$ axis) represent each embedding dimension. We demonstrate the utility of these Explain matrices and the differences seen when using different sets of sense features. (A)-(B) Explain matrices for the pointed nodes using positional sense features (shown on the $x$ axis). (C)-(D) Explain matrices for the same two nodes, but using the structural sense features (shown on the $x$ axis). Observe sub plot (C) where degree and average neighbor clustering stand out for the bridge nodes (green) and sub plot (D) where features like clustering coefficient and average neighbor degree stand out for the other nodes (grey). In the case of the structural sense features, all grey nodes would have similar Explain matrices given that they would all have similar sense features (i.e. degree, clustering coefficient etc..), but if passed in positional sense features instead, the grey nodes in different cliques would have different Explain matrices, as seen in sub plot (B) where the feature "Hops To Anchor In Bottom Clique" stands out.
• Figure 2: (A) Karate Club Network karate. (B) We embed the Karate Club network into 16 dimensions using DGI and visualize it in 2 dimensions using UMAP mcinnes2018umap. Explain matrix for (C) the instructor (node 1). (D) Explain matrix for a random student node (node 12). (E) Explain matrix for the president (node 34). Examining the Explain matrices for each of these nodes gives us an understanding of their placement in the embeddings space. Observe how features such as degree, personalized page rank etc., stand out for nodes 1 and 34 (subplots (C) and (E)) and features like average neighbor degree stands out for the random student node (subplot (D)).
• Figure 3: Explain matrices for the Karate Club network karate generated using DGI+XM for (A) the instructor (node 1), (B) a random student (node 12), and (C) the president (node 34). Compare each of (A), (B), and (C) to Figure \ref{['fig:figure_2']}(C),(D), and (E), respectively. Observe that the Explain matrices in this figure (generated by DGI+XM) are sparser and each sense feature is explained by fewer dimensions, when compared to the Explain matrices from Figure \ref{['fig:figure_2']} (generated by the standard DGI).
• Figure 4: Distribution of nuclear norms of the Explain matrices for each node for SDNE and SDNE+XM. Observe how the distribution of SDNE+XM is shifted to the left with a lower mean across each of the 6 datasets. See Section \ref{['nuclear_norm_distribution_section']} in the appendix for results on the other methods.
• Figure 5: AUC scores for link prediction across 4 embedding algorithms - SDNE, LINE, DGI, GMI - shown in each subplot, on the 6 networks defined in Table \ref{['table:real_nets']}. Error bars show standard error. AUC scores for the XM variants are comparable to the original algorithms with nuclear norms of the XM variants being lower. The XM variants also follow the original algorithms in terms of standard error, for example, SDNE has large variance in AUC for the PubMed dataset, which can also be seen for SDNE+XM. Results are from a three-fold cross validation experiment with 128 dimensional embeddings.