LightPROF: A Lightweight Reasoning Framework for Large Language Model on Knowledge Graph

TL;DR

LightPROF introduces a lightweight, parameter-efficient KGQA framework that enables small open-source LLMs to reason over Knowledge Graphs without retraining the LLM. It builds a Retrieve-Embed-Reason pipeline where a Transformer-based Knowledge Adapter converts textual and structural KG information into a compact knowledge soft prompt (p_s) fed to a frozen LLM via hard prompts. Through staged retrieval and fine-grained embedding of multi-hop KG paths, LightPROF achieves superior accuracy on WebQSP and CWQ while reducing input tokens and reasoning time compared to larger models and text-only approaches. The framework demonstrates strong plug-and-play compatibility with multiple LLM backbones and KG sources, offering practical gains in efficiency and scalability for KGQA tasks.

Abstract

Large Language Models (LLMs) have impressive capabilities in text understanding and zero-shot reasoning. However, delays in knowledge updates may cause them to reason incorrectly or produce harmful results. Knowledge Graphs (KGs) provide rich and reliable contextual information for the reasoning process of LLMs by structurally organizing and connecting a wide range of entities and relations. Existing KG-based LLM reasoning methods only inject KGs' knowledge into prompts in a textual form, ignoring its structural information. Moreover, they mostly rely on close-source models or open-source models with large parameters, which poses challenges to high resource consumption. To address this, we propose a novel Lightweight and efficient Prompt learning-ReasOning Framework for KGQA (LightPROF), which leverages the full potential of LLMs to tackle complex reasoning tasks in a parameter-efficient manner. Specifically, LightPROF follows a "Retrieve-Embed-Reason process", first accurately, and stably retrieving the corresponding reasoning graph from the KG through retrieval module. Next, through a Transformer-based Knowledge Adapter, it finely extracts and integrates factual and structural information from the KG, then maps this information to the LLM's token embedding space, creating an LLM-friendly prompt to be used by the LLM for the final reasoning. Additionally, LightPROF only requires training Knowledge Adapter and can be compatible with any open-source LLM. Extensive experiments on two public KGQA benchmarks demonstrate that LightPROF achieves superior performance with small-scale LLMs. Furthermore, LightPROF shows significant advantages in terms of input token count and reasoning time.

Figures (3)

• Figure 1: The architecture of our proposed Retrieve-Embed-Reason framework for knowledge graph question answer.
• Figure 2: Three Steps Retrieval Module, including: semantic extraction, relation retrieval, and reasoning graph sampling.
• Figure 3: Illustration of the Knowledge Adapter and the schematic representation of its crucial components.