Modeling Relational Data with Graph Convolutional Networks

TL;DR

This work introduces Relational Graph Convolutional Networks (R-GCNs) to model multi-relational knowledge graphs for link prediction and entity classification. It presents an encoder-decoder framework where relation-aware graph convolutions generate entity representations fed into a DistMult decoder for predicting edges, and a softmax classifier for node labeling. Regularization techniques address the proliferation of relation types, and empirical results demonstrate competitive performance, with a notable 29.8% improvement on FB15k-237 when enriching factorization models with an R-GCN encoder. The findings highlight the value of explicit neighborhood modeling in relational data and point to promising extensions, including alternative decoders, attention mechanisms, and scalable training strategies.

Abstract

Knowledge graphs enable a wide variety of applications, including question answering and information retrieval. Despite the great effort invested in their creation and maintenance, even the largest (e.g., Yago, DBPedia or Wikidata) remain incomplete. We introduce Relational Graph Convolutional Networks (R-GCNs) and apply them to two standard knowledge base completion tasks: Link prediction (recovery of missing facts, i.e. subject-predicate-object triples) and entity classification (recovery of missing entity attributes). R-GCNs are related to a recent class of neural networks operating on graphs, and are developed specifically to deal with the highly multi-relational data characteristic of realistic knowledge bases. We demonstrate the effectiveness of R-GCNs as a stand-alone model for entity classification. We further show that factorization models for link prediction such as DistMult can be significantly improved by enriching them with an encoder model to accumulate evidence over multiple inference steps in the relational graph, demonstrating a large improvement of 29.8% on FB15k-237 over a decoder-only baseline.

Figures (4)

• Figure 1: A knowledge base fragment: The nodes are entities, the edges are relations labeled with their types, the nodes are labeled with entity types (e.g., university). The edge and the node label shown in red are the missing information to be inferred.
• Figure 2: Diagram for computing the update of a single graph node/entity (red) in the R-GCN model. Activations ($d$-dimensional vectors) from neighboring nodes (dark blue) are gathered and then transformed for each relation type individually (for both in- and outgoing edges). The resulting representation (green) is accumulated in a (normalized) sum and passed through an activation function (such as the ReLU). This per-node update can be computed in parallel with shared parameters across the whole graph.
• Figure 3: (a) Depiction of an R-GCN model for entity classification with a per-node loss function. (b) Link prediction model with an R-GCN encoder (interspersed with fully-connected/dense layers) and a DistMult decoder that takes pairs of hidden node representations and produces a score for every (potential) edge in the graph. The loss is evaluated per edge.
• Figure 4: Mean reciprocal rank (MRR) for R-GCN and DistMult on the FB15k validation data as a function of the node degree (average of subject and object).