CYCLE: Cross-Year Contrastive Learning in Entity-Linking

TL;DR

Temporal degradation in Entity Linking arises as knowledge graphs evolve, hindering stable disambiguation over time. CYCLE introduces cross-year contrastive learning that leverages newly added and removed graph relations as positive and negative samples, integrated with a Bi-encoder for text and graph embeddings from relation and feature graphs. The model optimizes a joint objective L = a L_e + b L_f + c L_r, and its GCL-TempEL dataset extends TempEL with Wikidata5M to capture cross-year dynamics. Empirical results show substantial improvements over BLINK and SpEL, especially with larger temporal gaps and for low-degree entities, while maintaining competitive performance on static datasets, underscoring its practical impact for continually evolving KGs.

Abstract

Knowledge graphs constantly evolve with new entities emerging, existing definitions being revised, and entity relationships changing. These changes lead to temporal degradation in entity linking models, characterized as a decline in model performance over time. To address this issue, we propose leveraging graph relationships to aggregate information from neighboring entities across different time periods. This approach enhances the ability to distinguish similar entities over time, thereby minimizing the impact of temporal degradation. We introduce \textbf{CYCLE}: \textbf{C}ross-\textbf{Y}ear \textbf{C}ontrastive \textbf{L}earning for \textbf{E}ntity-Linking. This model employs a novel graph contrastive learning method to tackle temporal performance degradation in entity linking tasks. Our contrastive learning method treats newly added graph relationships as \textit{positive} samples and newly removed ones as \textit{negative} samples. This approach helps our model effectively prevent temporal degradation, achieving a 13.90\% performance improvement over the state-of-the-art from 2023 when the time gap is one year, and a 17.79\% improvement as the gap expands to three years. Further analysis shows that CYCLE is particularly robust for low-degree entities, which are less resistant to temporal degradation due to their sparse connectivity, making them particularly suitable for our method. The code and data are made available at \url{https://github.com/pengyu-zhang/CYCLE-Cross-Year-Contrastive-Learning-in-Entity-Linking}.

Figures (7)

• Figure 1: Entity linking (EL) connects text mentions to specific entities in a KG. In the example of this figure, the mention 'Amazon' could refer to Amazon rainforest, Amazon company, or mythical Amazons warriors entities. EL disambiguates this mention considering the surrounding mention context to pinpoint the correct entity: Amazon company.
• Figure 2: This is an example of learning the embedding of the entity Amazon (company) in 2022. Gray nodes represent entities that do not exist at a specific point in time. A (green) or (red) circle represents the use of a node as a positive or negative sample, respectively. In 2019, Amazon (company) had three neighbors: e-commerce, e-book reader, and bookstore. By 2022, while e-commerce continued as a neighbor, e-book reader and bookstore (negative samples) were replaced by digital streaming and cloud computing (positive samples). This shift indicates a change in Amazon (company)'s focus.
• Figure 3: The dataset construction process. We use Wikidata5M to extend TempEL with strutured graph representations. For each year, we identify newly added and removed edges for a target entity. Furthermore, we extract feature graph and feature matrix based on the textual description of the target entity. The green section represents the input to our model.
• Figure 4: The proposed CYCLE architecture leverages both text-based (left) and graph-based (right) inputs. Additionally, it introduces a novel Graph Contrastive Learning Module to efficiently adapt entity representations to temporal changes in the graph-based inputs. The figure illustrates positive (green) and negative (red) samples used in this Graph Contrastive Learning Module to capture temporal changes in graph-based inputs of the year 2022 with respect to those of the year 2019 for entity $e_i$.
• Figure 5: Recall@1 performance over 3-year gaps shows CYCLE outperforming both the BLINK and the SpEL on continual and new entities. Performance gains increase with larger temporal gaps, highlighting the robustness of our approach.