OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs

TL;DR

OGB-LSC addresses the gap in large-scale graph ML benchmarks by introducing MAG240M, WikiKG90M, and PCQM4M, each targeting node-, link-, and graph-level tasks on industry-scale graphs. The study demonstrates that expressive GNNs outperform simpler baselines when trained on massive data, and it documents the KDD Cup 2021 outcomes and techniques. Post-competition updates (WikiKG90Mv2 and PCQM4Mv2) improve realism and challenge, while public leaderboards foster ongoing community engagement. Collectively, OGB-LSC provides a practical, scalable framework to advance state-of-the-art in large-scale graph learning and encourages further methodological progress.

Abstract

Enabling effective and efficient machine learning (ML) over large-scale graph data (e.g., graphs with billions of edges) can have a great impact on both industrial and scientific applications. However, existing efforts to advance large-scale graph ML have been largely limited by the lack of a suitable public benchmark. Here we present OGB Large-Scale Challenge (OGB-LSC), a collection of three real-world datasets for facilitating the advancements in large-scale graph ML. The OGB-LSC datasets are orders of magnitude larger than existing ones, covering three core graph learning tasks -- link prediction, graph regression, and node classification. Furthermore, we provide dedicated baseline experiments, scaling up expressive graph ML models to the massive datasets. We show that expressive models significantly outperform simple scalable baselines, indicating an opportunity for dedicated efforts to further improve graph ML at scale. Moreover, OGB-LSC datasets were deployed at ACM KDD Cup 2021 and attracted more than 500 team registrations globally, during which significant performance improvements were made by a variety of innovative techniques. We summarize the common techniques used by the winning solutions and highlight the current best practices in large-scale graph ML. Finally, we describe how we have updated the datasets after the KDD Cup to further facilitate research advances. The OGB-LSC datasets, baseline code, and all the information about the KDD Cup are available at https://ogb.stanford.edu/docs/lsc/ .

Figures (8)

• Figure 1: Basic statistics of the OGB-LSC datasets used in KDD Cup 2021. Datasets marked by $\dagger$ has been updated to v2 after the KDD Cup (cf. Section \ref{['sec:update']}).
• Figure 1: Overview of the three OGB-LSC datasets, covering node-, link-, and graph-level prediction tasks, respectively.(a)MAG240M is a heterogeneous academic graph, and the task is to predict the subject areas of papers situated in the heterogeneous graph (cf. Section \ref{['subsec:node']}). (b)WikiKG90M is a knowledge graph, and the task is to impute missing triplets (cf. Section \ref{['subsec:link']}). (c)PCQM4M is a quantum chemistry dataset, and the task is to predict an important molecular property---the HOMO-LUMO gap---of a given molecule (cf. Section \ref{['subsec:graph']}).
• Figure 2: A schema diagram of MAG240M.
• Figure 4: Textual representation of validation triplets whose head entities only appear once as head in the training WikiKG90M.
• Figure 6: Results of PCQM4M measured by MAE [eV]. The lower, the better. Ablation study of using only 10% of training data is also shown. Chemical accuracy indicates the final goal for practical usefulness.