Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs

Jiejun Tan, Zhicheng Dou, Yutao Zhu, Peidong Guo, Kun Fang, Ji-Rong Wen

TL;DR

<3-5 sentence high-level summary> Open-domain QA often struggles with LLM hallucinations and costly retrieval. SlimPLM introduces a slim proxy-LM to generate heuristic answers, then uses a retrieval-necessity judgment module and claim-based query filtering to decide when and what to retrieve. The approach integrates with RAG as a plug-in, reducing LLM inference costs while maintaining or improving end-to-end QA performance across five datasets. Key contributions include (1) a proxy-driven retrieval necessity signal, (2) heuristic-driven query rewriting, and (3) a claim-based filtering mechanism for efficient retrieval. The empirical results demonstrate competitive or state-of-the-art performance with lower computational overhead compared to baselines.

Abstract

The integration of large language models (LLMs) and search engines represents a significant evolution in knowledge acquisition methodologies. However, determining the knowledge that an LLM already possesses and the knowledge that requires the help of a search engine remains an unresolved issue. Most existing methods solve this problem through the results of preliminary answers or reasoning done by the LLM itself, but this incurs excessively high computational costs. This paper introduces a novel collaborative approach, namely SlimPLM, that detects missing knowledge in LLMs with a slim proxy model, to enhance the LLM's knowledge acquisition process. We employ a proxy model which has far fewer parameters, and take its answers as heuristic answers. Heuristic answers are then utilized to predict the knowledge required to answer the user question, as well as the known and unknown knowledge within the LLM. We only conduct retrieval for the missing knowledge in questions that the LLM does not know. Extensive experimental results on five datasets with two LLMs demonstrate a notable improvement in the end-to-end performance of LLMs in question-answering tasks, achieving or surpassing current state-of-the-art models with lower LLM inference costs.

Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs

TL;DR

<3-5 sentence high-level summary> Open-domain QA often struggles with LLM hallucinations and costly retrieval. SlimPLM introduces a slim proxy-LM to generate heuristic answers, then uses a retrieval-necessity judgment module and claim-based query filtering to decide when and what to retrieve. The approach integrates with RAG as a plug-in, reducing LLM inference costs while maintaining or improving end-to-end QA performance across five datasets. Key contributions include (1) a proxy-driven retrieval necessity signal, (2) heuristic-driven query rewriting, and (3) a claim-based filtering mechanism for efficient retrieval. The empirical results demonstrate competitive or state-of-the-art performance with lower computational overhead compared to baselines.

Abstract

The integration of large language models (LLMs) and search engines represents a significant evolution in knowledge acquisition methodologies. However, determining the knowledge that an LLM already possesses and the knowledge that requires the help of a search engine remains an unresolved issue. Most existing methods solve this problem through the results of preliminary answers or reasoning done by the LLM itself, but this incurs excessively high computational costs. This paper introduces a novel collaborative approach, namely SlimPLM, that detects missing knowledge in LLMs with a slim proxy model, to enhance the LLM's knowledge acquisition process. We employ a proxy model which has far fewer parameters, and take its answers as heuristic answers. Heuristic answers are then utilized to predict the knowledge required to answer the user question, as well as the known and unknown knowledge within the LLM. We only conduct retrieval for the missing knowledge in questions that the LLM does not know. Extensive experimental results on five datasets with two LLMs demonstrate a notable improvement in the end-to-end performance of LLMs in question-answering tasks, achieving or surpassing current state-of-the-art models with lower LLM inference costs.
Paper Structure (28 sections, 4 equations, 3 figures, 10 tables)

This paper contains 28 sections, 4 equations, 3 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Retrieval-Augmented Generation (RAG)
  4. Retrieval Necessity Judgment
  5. Query Formulation
  6. Methodology
  7. Problem Formulation & Framework
  8. Retrieval Necessity Judgment
  9. Judgment Model
  10. Judgment Label Collection
  11. Retrieval Target Determination
  12. Heuristic Answer-Driven Query Rewrite
  13. Claim-based Query Filter
  14. Experiments
  15. Datasets
  16. ...and 13 more sections

Figures (3)

  • Figure 1: A display of the main process of SlimPLM. Solid lines with arrows represent the flow of data, while dashed lines with arrows signify control signals from the retrieval necessity judgment model. Step 1 and step 2 are mandatory in the pipeline, but step 3 involves choosing between direct generation and RAG.
  • Figure 2: The rouge performance for with and without retrieval on ELI5 dataset.
  • Figure 3: The proportion of samples ($y$-axis) with EM scores higher than certain values ($x$-axis).