Label Informed Contrastive Pretraining for Node Importance Estimation on Knowledge Graphs

TL;DR

This work tackles Node Importance Estimation on knowledge graphs by introducing LICAP, a pretraining framework that injects prior knowledge about higher-importance nodes. LICAP converts a regression problem into a classification-like task via label-informed grouping and uses a two-level top-nodes hierarchical sampling with two InfoNCE-style losses, aided by a predicate-aware GAT (PreGAT) for KG edges. Empirical results across three real-world KGs show LICAP pretrained embeddings boost downstream NIE methods, achieving new state-of-the-art on both regression and ranking metrics, and ablations validate the effectiveness of its sampling strategy and PredGAT. The framework is pluggable with existing NIE models, demonstrating broad applicability and potential for extension to dynamic graphs and other graph-based regression/ranking tasks.

Abstract

Node Importance Estimation (NIE) is a task of inferring importance scores of the nodes in a graph. Due to the availability of richer data and knowledge, recent research interests of NIE have been dedicating to knowledge graphs for predicting future or missing node importance scores. Existing state-of-the-art NIE methods train the model by available labels, and they consider every interested node equally before training. However, the nodes with higher importance often require or receive more attention in real-world scenarios, e.g., people may care more about the movies or webpages with higher importance. To this end, we introduce Label Informed ContrAstive Pretraining (LICAP) to the NIE problem for being better aware of the nodes with high importance scores. Specifically, LICAP is a novel type of contrastive learning framework that aims to fully utilize the continuous labels to generate contrastive samples for pretraining embeddings. Considering the NIE problem, LICAP adopts a novel sampling strategy called top nodes preferred hierarchical sampling to first group all interested nodes into a top bin and a non-top bin based on node importance scores, and then divide the nodes within top bin into several finer bins also based on the scores. The contrastive samples are generated from those bins, and are then used to pretrain node embeddings of knowledge graphs via a newly proposed Predicate-aware Graph Attention Networks (PreGAT), so as to better separate the top nodes from non-top nodes, and distinguish the top nodes within top bin by keeping the relative order among finer bins. Extensive experiments demonstrate that the LICAP pretrained embeddings can further boost the performance of existing NIE methods and achieve the new state-of-the-art performance regarding both regression and ranking metrics. The source code for reproducibility is available at https://github.com/zhangtia16/LICAP

Figures (7)

• Figure 1: The toy example of a movie KG. Edges with different colors indicate different edge types. Node importance of movies is defined by its popularity score. The movie (red) with higher importance often receives more attention.
• Figure 2: An overview of the proposed framework.
• Figure 3: The contrastive samples and loss for the top bin and non-top bin to better separate top nodes from non-top nodes.
• Figure 4: The contrastive samples and loss for the finer bins within top bin to better distinguish top nodes by keeping relative orders among the finer bins.
• Figure 5: Predicate-aware GAT (PreGAT) to inherently include predicates (i.e., edge types containing relational information) in knowledge graphs.