Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Supportiveness-based Knowledge Rewriting for Retrieval-augmented Language Modeling

Zile Qiao, Wei Ye, Yong Jiang, Tong Mo, Pengjun Xie, Weiping Li, Fei Huang, Shikun Zhang

TL;DR

This paper introduces Supportiveness-based Knowledge Rewriting (SKR), a robust and pluggable knowledge rewriter inherently optimized for LLM generation, and introduces the novel concept of "supportiveness"--which represents how effectively a knowledge piece facilitates downstream tasks--by considering the perplexity impact of augmented knowledge on the response text of a white-box LLM.

Abstract

Retrieval-augmented language models (RALMs) have recently shown great potential in mitigating the limitations of implicit knowledge in LLMs, such as untimely updating of the latest expertise and unreliable retention of long-tail knowledge. However, since the external knowledge base, as well as the retriever, can not guarantee reliability, potentially leading to the knowledge retrieved not being helpful or even misleading for LLM generation. In this paper, we introduce Supportiveness-based Knowledge Rewriting (SKR), a robust and pluggable knowledge rewriter inherently optimized for LLM generation. Specifically, we introduce the novel concept of "supportiveness"--which represents how effectively a knowledge piece facilitates downstream tasks--by considering the perplexity impact of augmented knowledge on the response text of a white-box LLM. Based on knowledge supportiveness, we first design a training data curation strategy for our rewriter model, effectively identifying and filtering out poor or irrelevant rewrites (e.g., with low supportiveness scores) to improve data efficacy. We then introduce the direct preference optimization (DPO) algorithm to align the generated rewrites to optimal supportiveness, guiding the rewriter model to summarize augmented content that better improves the final response. Comprehensive evaluations across six popular knowledge-intensive tasks and four LLMs have demonstrated the effectiveness and superiority of SKR. With only 7B parameters, SKR has shown better knowledge rewriting capability over GPT-4, the current state-of-the-art general-purpose LLM.

Supportiveness-based Knowledge Rewriting for Retrieval-augmented Language Modeling

TL;DR

This paper introduces Supportiveness-based Knowledge Rewriting (SKR), a robust and pluggable knowledge rewriter inherently optimized for LLM generation, and introduces the novel concept of "supportiveness"--which represents how effectively a knowledge piece facilitates downstream tasks--by considering the perplexity impact of augmented knowledge on the response text of a white-box LLM.

Abstract

Retrieval-augmented language models (RALMs) have recently shown great potential in mitigating the limitations of implicit knowledge in LLMs, such as untimely updating of the latest expertise and unreliable retention of long-tail knowledge. However, since the external knowledge base, as well as the retriever, can not guarantee reliability, potentially leading to the knowledge retrieved not being helpful or even misleading for LLM generation. In this paper, we introduce Supportiveness-based Knowledge Rewriting (SKR), a robust and pluggable knowledge rewriter inherently optimized for LLM generation. Specifically, we introduce the novel concept of "supportiveness"--which represents how effectively a knowledge piece facilitates downstream tasks--by considering the perplexity impact of augmented knowledge on the response text of a white-box LLM. Based on knowledge supportiveness, we first design a training data curation strategy for our rewriter model, effectively identifying and filtering out poor or irrelevant rewrites (e.g., with low supportiveness scores) to improve data efficacy. We then introduce the direct preference optimization (DPO) algorithm to align the generated rewrites to optimal supportiveness, guiding the rewriter model to summarize augmented content that better improves the final response. Comprehensive evaluations across six popular knowledge-intensive tasks and four LLMs have demonstrated the effectiveness and superiority of SKR. With only 7B parameters, SKR has shown better knowledge rewriting capability over GPT-4, the current state-of-the-art general-purpose LLM.
Paper Structure (24 sections, 8 equations, 5 figures, 9 tables)

This paper contains 24 sections, 8 equations, 5 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Architecture
  4. Supportiveness
  5. Supportiveness-based Rewrite Data Generation
  6. Automated Rewrite Data Generation
  7. Supportiveness-based Data Refinement
  8. Supportiveness-guided Alignment
  9. Experiment Setup
  10. Implementation Details
  11. Evaluation
  12. Experiment Results
  13. Main Results
  14. Impact of Noise and Misleading Information
  15. Effects on Compression Rate
  16. ...and 9 more sections

Figures (5)

  • Figure 1: To measure the change in perplexity induced by correct knowledge, we sampled 500 correctly answered questions (True set) and 500 incorrectly answered questions (False set) from TriviaQA. We then calculate the perplexity before and after introducing retrieved knowledge, where the retrieved knowledge included the correct answers.
  • Figure 2: A visual illustration of the proposed Supportiveness-based Rewrite Data Generation and Supportiveness-guided Alignment. "pp" represents perplexity, and "ss" denotes the supportiveness score.
  • Figure 3: Prompt for GPT-4.
  • Figure 4: Prompts for calculating supportiveness.
  • Figure 5: Modified prompt for GPT-4.