Revisiting Semi-Supervised Learning with Graph Embeddings

TL;DR

The paper addresses semi-supervised learning on graph-structured data by learning node embeddings that jointly predict labels and graph context. It introduces Planetoid, a neural-network framework with transductive and inductive variants, optimizing a combined supervised and unsupervised (context-prediction) objective. Empirical results across text classification and entity-annotation tasks show substantial improvements over prior graph-based and embedding approaches, with notable gains in the inductive setting when features are informative. This work provides a scalable approach to integrate embedding learning with task-specific supervision in graph-structured domains.

Abstract

We present a semi-supervised learning framework based on graph embeddings. Given a graph between instances, we train an embedding for each instance to jointly predict the class label and the neighborhood context in the graph. We develop both transductive and inductive variants of our method. In the transductive variant of our method, the class labels are determined by both the learned embeddings and input feature vectors, while in the inductive variant, the embeddings are defined as a parametric function of the feature vectors, so predictions can be made on instances not seen during training. On a large and diverse set of benchmark tasks, including text classification, distantly supervised entity extraction, and entity classification, we show improved performance over many of the existing models.

Figures (3)

• Figure 1: An example of sampling from context distribution $p(i, c, \gamma)$ when $\gamma = 1$ and $d = 2$. In circles, $+1$ denotes positive instances, $-1$ denotes negative instances, and $?$ denotes unlabeled instances. If $random < r_2$, we first sample a random walk $2 \rightarrow 1 \rightarrow 4 \rightarrow 6$, and then sample two nodes in the random walk within distance $d$. If $random \geq r_2$, we sample two instances with the same labels.
• Figure 2: Network architecture: transductive v.s. inductive. Each dotted arrow represents a feed-forward network with an arbitrary number of layers (we use only one layer in our experiments). Solid arrows denote direct connections.
• Figure 3: t-SNE Visualization of embedding spaces on the Cora dataset. Each color denotes a class.