Node Embeddings via Neighbor Embeddings

TL;DR

Graph NE introduces a non-parametric neighbor-embedding framework that embeds graph nodes by directly pulling adjacent nodes together using affinities derived from the graph itself, eliminating the need for random walks. The approach unifies high-dimensional node embeddings and 2D graph layouts by linking the InfoNCE objective to KL divergence, enabling a $d=128$ embedding and a graph-$t$-SNE style visualization via a CNE backend. Empirically, Graph NE consistently outperforms state-of-the-art non-parametric methods on local structure preservation across eight graphs and delivers superior 2D layouts, with competitive classification results and strong local structure metrics. The method is simple, scalable (linear in edges), and feature-free, with promising directions including parametric extensions, alternative neighbor-embedding backends like UMAP, and systematic study of the temperature parameter $\tau$.

Abstract

Node embeddings are a paradigm in non-parametric graph representation learning, where graph nodes are embedded into a given vector space to enable downstream processing. State-of-the-art node-embedding algorithms, such as DeepWalk and node2vec, are based on random-walk notions of node similarity and on contrastive learning. In this work, we introduce the graph neighbor-embedding (graph NE) framework that directly pulls together embedding vectors of adjacent nodes without relying on any random walks. We show that graph NE strongly outperforms state-of-the-art node-embedding algorithms in terms of local structure preservation. Furthermore, we apply graph NE to the 2D node-embedding problem, obtaining graph t-SNE layouts that also outperform existing graph-layout algorithms.

Figures (8)

• Figure 1: Graph $G=(\mathcal{V}, \mathcal{E})$ embedded into $S^{127}$ and $\mathbb R^2$ with graph NE. Blue denotes the attractive force between neighboring nodes $i$ and $j$ with $(i,j)\in\mathcal{E}$, orange corresponds to repulsive forces between all points.
• Figure 2: Learning dynamics of the 128-dimensional CNE embeddings of nodes in a stochastic-block-model graph with 10 blocks. (a, b)$t$-SNE visualizations of the 128D CNE embeddings with $\tau=0.05$, during the first epoch and after ten epochs. (c) The neighbor recall as a function of the training epoch, for $\tau=0.05$ and for $\tau=0.5$. Labeled points correspond to $t$-SNE visualizations left/right. (d, e) Same as (a, b), but for $\tau=0.5$.
• Figure 3: Performance metrics for node embeddings: (a) neighbor recall, (b)$k$NN accuracy, (c) linear accuracy. Datasets are ordered by the number of edges. For node2vec we did a grid search over $p,q\in\{0.25, 0.5, 1, 2, 4\}$ (\ref{['fig:node2vec-pq']}) and show results with the highest neighbor recall.
• Figure 5: Embeddings of the Computer and Photo datasets obtained using our graph NE (graph $t$-SNE), DRGraph, ForceAtlas2, and $t$-FDP. Embeddings were aligned using Procrustes rotation. See Figure \ref{['fig:all-embeddings']} for all datasets and methods.
• Figure S1: Computation times. All computations were performed on a cluster which isolates the computing resources and removes interference between concurrent computations. All 2D experiments require only CPUs and were ran on 8 cores of an Intel Xeon Gold 6226R. Experiments in 128D ran on a single Nvidia 2080ti GPU card. For node2vec, this shows runtime for $p=q=1$; we ran 25 parameter combinations (\ref{['fig:node2vec-pq']}), so our actual runtime including hyperparameter tuning was much larger.

Theorems & Definitions (3)

• Theorem 4.1: label=thm:infonce
• Theorem A.1: label=thm:infonce
• proof