Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Retrieval-Feedback-Driven Distillation and Preference Alignment for Efficient LLM-based Query Expansion

Minghan Li, Guodong Zhou

Abstract

Large language models have recently enabled a generative paradigm for query expansion, but their high inference cost makes direct deployment difficult in practical retrieval systems. To address this issue, a retrieval-feedback-driven distillation and preference-alignment framework is proposed to transfer retrieval-friendly expansion behavior from a strong teacher model to a compact student model. Rather than relying on few-shot exemplars at inference time, the framework first leverages two complementary types of teacher-generated expansions, produced under zero-shot and few-shot prompting conditions, as supervision signals for distillation and as candidate pools for preference construction. A retrieval-metric-driven strategy is then introduced to automatically form chosen/rejected expansion pairs according to nDCG@10 differences, and Direct Preference Optimization is applied to explicitly align generation preferences with retrieval objectives. Experiments on TREC DL19/20/21 and MIRACL-zh show that the proposed approach preserves strong retrieval effectiveness while substantially reducing inference cost. In particular, the distilled Qwen3-4B model reaches about 97% of the teacher (DeepSeek-685B) model's nDCG@10 performance on DL19, and remains effective on the Chinese MIRACL-zh benchmark, demonstrating strong practicality across both English and Chinese retrieval settings.

Retrieval-Feedback-Driven Distillation and Preference Alignment for Efficient LLM-based Query Expansion

Abstract

Large language models have recently enabled a generative paradigm for query expansion, but their high inference cost makes direct deployment difficult in practical retrieval systems. To address this issue, a retrieval-feedback-driven distillation and preference-alignment framework is proposed to transfer retrieval-friendly expansion behavior from a strong teacher model to a compact student model. Rather than relying on few-shot exemplars at inference time, the framework first leverages two complementary types of teacher-generated expansions, produced under zero-shot and few-shot prompting conditions, as supervision signals for distillation and as candidate pools for preference construction. A retrieval-metric-driven strategy is then introduced to automatically form chosen/rejected expansion pairs according to nDCG@10 differences, and Direct Preference Optimization is applied to explicitly align generation preferences with retrieval objectives. Experiments on TREC DL19/20/21 and MIRACL-zh show that the proposed approach preserves strong retrieval effectiveness while substantially reducing inference cost. In particular, the distilled Qwen3-4B model reaches about 97% of the teacher (DeepSeek-685B) model's nDCG@10 performance on DL19, and remains effective on the Chinese MIRACL-zh benchmark, demonstrating strong practicality across both English and Chinese retrieval settings.
Paper Structure (36 sections, 4 equations, 6 figures, 10 tables)

This paper contains 36 sections, 4 equations, 6 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Query Expansion Methods
  4. Knowledge Distillation and Reinforcement Learning Alignment
  5. Preference-Aligned Knowledge Distillation for Query Expansion Based on Large Language Models
  6. Supervised Data Construction with Large Language Models
  7. Prompt Design
  8. Zero-shot Prompt Template
  9. Few-shot Prompt Template
  10. Supervised Fine-Tuning of the Student Model with Fewer Parameters
  11. Prompt Consistency and Input-Output Format
  12. Training Objective and Loss Function
  13. Preference-Aligned Knowledge Distillation
  14. Preference Data Construction (Generation of DPO Training Pairs)
  15. DPO Objective and Training Setup
  16. ...and 21 more sections

Figures (6)

  • Figure 1: Overall system architecture of the proposed method.
  • Figure 2: Zero-shot prompt template.
  • Figure 3: Few-shot prompt template with four in-context examples.
  • Figure 4: Peak GPU memory usage for query expansion generation under different settings on DL19 (local inference).
  • Figure 5: Effect of LoRA rank on nDCG@10 performance of the model after SFT on TREC DL19.
  • ...and 1 more figures