Large Language Model Enhanced Knowledge Representation Learning: A Survey

TL;DR

The paper surveys how Large Language Models (LLMs) can bolster Knowledge Representation Learning (KRL) for Knowledge Graphs by incorporating textual context to mitigate KG sparsity. It classifies methods into encoder-based, encoder-decoder-based, and decoder-based approaches, detailing representative models, architectures, and training losses, with emphasis on how textual signals and prompts complement structural information. Across multiple downstream tasks—entity typing, relation classification, relation/triple/link prediction—the surveyed studies show consistent performance gains from LLM augmentation, though gains vary by dataset and task, and benchmarking remains heterogeneous. The paper also maps six future directions, including dynamic multimodal knowledge representation, efficiency and explainability, graph-centric instruction tuning, and continual learning, highlighting practical implications for scalable, robust KG understanding. Overall, integrating LLMs with KRL holds promise for more accurate, context-aware knowledge representations and more capable downstream systems, especially in settings with long-tail entities and rich textual descriptions.

Abstract

Knowledge Representation Learning (KRL) is crucial for enabling applications of symbolic knowledge from Knowledge Graphs (KGs) to downstream tasks by projecting knowledge facts into vector spaces. Despite their effectiveness in modeling KG structural information, KRL methods are suffering from the sparseness of KGs. The rise of Large Language Models (LLMs) built on the Transformer architecture presents promising opportunities for enhancing KRL by incorporating textual information to address information sparsity in KGs. LLM-enhanced KRL methods, including three key approaches, encoder-based methods that leverage detailed contextual information, encoder-decoder-based methods that utilize a unified Seq2Seq model for comprehensive encoding and decoding, and decoder-based methods that utilize extensive knowledge from large corpora, have significantly advanced the effectiveness and generalization of KRL in addressing a wide range of downstream tasks. This work provides a broad overview of downstream tasks while simultaneously identifying emerging research directions in these evolving domains.

Figures (9)

• Figure 1: An overview of triple-based representation methods.
• Figure 2: An overview of translation encoding methods.
• Figure 3: An overview of independent representation methods.
• Figure 4: An overview of structure-based representation methods.
• Figure 5: An overview of fine-tuning methods.