KALE: Enhancing Knowledge Manipulation in Large Language Models via Knowledge-aware Learning

TL;DR

KALE addresses the knowledge manipulation gap in LLMs post-SFT by introducing Knowledge-Induced data Synthesis (KI) to generate high-quality rationales from knowledge graphs and Knowledge-Aware fine-tuning (KA) to align model outputs with and without rationales via a KL-divergence objective. The framework enables efficient reasoning-path extraction using a multi-path A* search over KGs and rationale generation with GPT-4o, followed by distribution alignment that allows robust knowledge manipulation at inference without relying on retrieved data. Empirical results across eight benchmarks and six backbones show KALE yields substantial gains (up to 11.72% and average 4.18%) over SFT baselines, with ablations confirming the necessity of both KI and KA components. The approach is practical, adds minimal inference overhead, and demonstrates strong generalization and data-efficiency, though it relies on structured data and KGs and invites future work on domain-specific KG construction and advanced linker strategies.

Abstract

Despite the impressive performance of large language models (LLMs) pretrained on vast knowledge corpora, advancing their knowledge manipulation-the ability to effectively recall, reason, and transfer relevant knowledge-remains challenging. Existing methods mainly leverage Supervised Fine-Tuning (SFT) on labeled datasets to enhance LLMs' knowledge manipulation ability. However, we observe that SFT models still exhibit the known&incorrect phenomenon, where they explicitly possess relevant knowledge for a given question but fail to leverage it for correct answers. To address this challenge, we propose KALE (Knowledge-Aware LEarning)-a post-training framework that leverages knowledge graphs (KGs) to generate high-quality rationales and enhance LLMs' knowledge manipulation ability. Specifically, KALE first introduces a Knowledge-Induced (KI) data synthesis method that efficiently extracts multi-hop reasoning paths from KGs to generate high-quality rationales for question-answer pairs. Then, KALE employs a Knowledge-Aware (KA) fine-tuning paradigm that enhances knowledge manipulation by internalizing rationale-guided reasoning through minimizing the KL divergence between predictions with and without rationales. Extensive experiments on eight popular benchmarks across six different LLMs demonstrate the effectiveness of KALE, achieving accuracy improvements of up to 11.72% and an average of 4.18%.

Figures (22)

• Figure 1: While both post-trained LLMs know relevant knowledge, the LLM via SFT still cannot recall the knowledge to answer. In contrast, KALE effectively recalls the knowledge and answers correctly. We use Mistral 7B mistral as an example, and more cases are in Appendix \ref{['app:more_case']}.
• Figure 2: An overview of KALE. For a given Q&A pair, the workflow of KALE is as follows. (1) Perform named entity recognition to extract relevant question and answer entities. (2) Search for reasoning paths via the proposed multi-path A* algorithm. (3) Combine reasoning paths with the Q&A pair to generate the corresponding rationales via GPT-4o. (4) Align the LLM's output distributions with and without rationales via knowledge-aware fine-tuning.
• Figure 3: Known&incorrect phenomenon analysis: following the known fact checking in Figure \ref{['fig:case']}, we collect cases where LLMs possess the knowledge to answer and analyze the ratios of correct and incorrect answers, denoted as known&correct and known&incorrect.
• Figure 4: Ratios of augmented rationales: by setting the data augmentation ratio from 5% to 50%, we explore the differences between KALE and the SFT under varying data scales. We provide results of LlaMA3 8B, Mistral 7B, and Qwen2.5 32B as the backbones as examples, with more results in Appendix \ref{['app:case_ratio']}.
• Figure 5: An example of known&incorrect phenomenon comparison using LlaMA3 8B as the backbone model.