Semantic-Condition Tuning: Fusing Graph Context with Large Language Models for Knowledge Graph Completion

TL;DR

Knowledge graphs are inherently incomplete, and naive fusion with large language models often relies on shallow prefix-tuning. The paper introduces Semantic-Condition Tuning (SCT), which combines a Semantic Graph Module that distills a context-aware semantic condition vector $c_S$ from local KG neighborhoods with a Condition-Adaptive Fusion Module that performs a FiLM-style affine transformation to yield a pre-fused embedding $X'$, before LLM processing. This deep, feature-level integration enables more accurate and robust KG reasoning, achieving state-of-the-art or highly competitive results on standard KGC benchmarks with parameter-efficient fine-tuning via LoRA. Ablation studies confirm that both semantic conditioning and adaptive fusion are essential, and analyses show the benefit of knowledge-enhanced, relation-centric semantics for guiding LLMs. The work suggests promising directions, including hierarchical semantic conditioning and temporal knowledge graphs, to further enhance structure-aware reasoning in LLM-based KGC.

Abstract

Fusing Knowledge Graphs with Large Language Models is crucial for knowledge-intensive tasks like knowledge graph completion. The prevailing paradigm, prefix-tuning, simply concatenates knowledge embeddings with text inputs. However, this shallow fusion overlooks the rich relational semantics within KGs and imposes a significant implicit reasoning burden on the LLM to correlate the prefix with the text. To address these, we propose Semantic-condition Tuning (SCT), a new knowledge injection paradigm comprising two key modules. First, a Semantic Graph Module employs a Graph Neural Network to extract a context-aware semantic condition from the local graph neighborhood, guided by knowledge-enhanced relations. Subsequently, this condition is passed to a Condition-Adaptive Fusion Module, which, in turn, adaptively modulates the textual embedding via two parameterized projectors, enabling a deep, feature-wise, and knowledge-aware interaction. The resulting pre-fused embedding is then fed into the LLM for fine-tuning. Extensive experiments on knowledge graph benchmarks demonstrate that SCT significantly outperforms prefix-tuning and other strong baselines. Our analysis confirms that by modulating the input representation with semantic graph context before LLM inference, SCT provides a more direct and potent signal, enabling more accurate and robust knowledge reasoning.

Figures (3)

• Figure 1: The guiding effect of structured information on LLMs. The meaning of "Treats" changes dynamically with its context. Unlike the weakly-related information from a shallow prefix, the highly-relevant semantic condition effectively guides the LLM to perform accurate reasoning.
• Figure 2: The SCT framework. Semantic Graph Module extracts a context-aware semantic condition from the input knowledge graph. This condition is then used by the Condition-Adaptive Fusion Module to perform a feature-wise modulation on the textual token embeddings, generating a knowledge-infused representation for the LLM to process.
• Figure 3: Sensitivity analysis of the Top-K parameter.