From communities to interpretable network and word embedding: an unified approach

TL;DR

The paper tackles the challenge of learning graph and word embeddings that are both effective and intrinsically interpretable, while also reducing computational burden. It introduces the Lower Dimension Bipartite Framework (LDBGF), a unifying theory that projects graphs into a low-dimensional bipartite structure to produce sparse, interpretable vertex embeddings; two implementations, SINr-NR and SINr-MF, derive embeddings from community structure detected by Louvain. Through extensive experiments across microscopic, mesoscopic, and macroscopic scales, and with a concrete NLP application to word co-occurrence networks, the authors demonstrate that SINr-NR offers strong performance, robustness, and interpretability, often rivaling or approaching baselines like Word2Vec while maintaining efficiency. The approach emphasizes intrinsic interpretability (dimensions tied to communities) and stability, and shows practical potential for auditing and fair deployment in real-world settings. The work suggests future extensions to directed and temporal networks, aiming to broaden applicability while preserving interpretability and efficiency.

Abstract

Modelling information from complex systems such as humans social interaction or words co-occurrences in our languages can help to understand how these systems are organized and function. Such systems can be modelled by networks, and network theory provides a useful set of methods to analyze them. Among these methods, graph embedding is a powerful tool to summarize the interactions and topology of a network in a vectorized feature space. When used in input of machine learning algorithms, embedding vectors help with common graph problems such as link prediction, graph matching, etc. Word embedding has the goal of representing the sense of words, extracting it from large text corpora. Despite differences in the structure of information in input of embedding algorithms, many graph embedding approaches are adapted and inspired from methods in NLP. Limits of these methods are observed in both domains. Most of these methods require long and resource greedy training. Another downside to most methods is that they are black-box, from which understanding how the information is structured is rather complex. Interpretability of a model allows understanding how the vector space is structured without the need for external information, and thus can be audited more easily. With both these limitations in mind, we propose a novel framework to efficiently embed network vertices in an interpretable vector space. Our Lower Dimension Bipartite Framework (LDBGF) leverages the bipartite projection of a network using cliques to reduce dimensionality. Along with LDBGF, we introduce two implementations of this framework that rely on communities instead of cliques: SINr-NR and SINr-MF. We show that SINr-MF can perform well on classical graphs and SINr-NR can produce high-quality graph and word embeddings that are interpretable and stable across runs.

Figures (11)

• Figure 1: An airport network of the United States of America (size of vertices proportional to their degrees).
• Figure 2: Flights connected to Albany and Columbus and distribution of connections towards states.
• Figure 3: Illustration of the LDBGF, vertices are linked to the cliques they belong to.
• Figure 4: Illustration of SINr-NR, vertices are represented based on the communities they are linked to.
• Figure 5: Average accuracy (over 50 runs) on Link Prediction according to $\gamma$ with SINr-NR in yellow and SINr-MF in blue for Cora, Email-EU and ArXiv. Average number of dimensions of our models (over 50 runs) according to the $\gamma$ is given in (d).