Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ReFeed: Retrieval Feedback-Guided Dataset Construction for Style-Aware Query Rewriting

Jiyoon Myung, Jungki Son, Kyungro Lee, Jihyeon Park, Joohyung Han

TL;DR

This work introduces a retrieval feedback-driven dataset generation framework that automatically identifies failed retrieval cases, leverages large language models to rewrite queries in the style of relevant documents, and verifies improvement through re-retrieval.

Abstract

Retrieval systems often fail when user queries differ stylistically or semantically from the language used in domain documents. Query rewriting has been proposed to bridge this gap, improving retrieval by reformulating user queries into semantically equivalent forms. However, most existing approaches overlook the stylistic characteristics of target documents-their domain-specific phrasing, tone, and structure-which are crucial for matching real-world data distributions. We introduce a retrieval feedback-driven dataset generation framework that automatically identifies failed retrieval cases, leverages large language models to rewrite queries in the style of relevant documents, and verifies improvement through re-retrieval. The resulting corpus of (original, rewritten) query pairs enables the training of rewriter models that are explicitly aware of document style and retrieval feedback. This work highlights a new direction in data-centric information retrieval, emphasizing how feedback loops and document-style alignment can enhance the reasoning and adaptability of RAG systems in real-world, domain-specific contexts.

ReFeed: Retrieval Feedback-Guided Dataset Construction for Style-Aware Query Rewriting

TL;DR

This work introduces a retrieval feedback-driven dataset generation framework that automatically identifies failed retrieval cases, leverages large language models to rewrite queries in the style of relevant documents, and verifies improvement through re-retrieval.

Abstract

Retrieval systems often fail when user queries differ stylistically or semantically from the language used in domain documents. Query rewriting has been proposed to bridge this gap, improving retrieval by reformulating user queries into semantically equivalent forms. However, most existing approaches overlook the stylistic characteristics of target documents-their domain-specific phrasing, tone, and structure-which are crucial for matching real-world data distributions. We introduce a retrieval feedback-driven dataset generation framework that automatically identifies failed retrieval cases, leverages large language models to rewrite queries in the style of relevant documents, and verifies improvement through re-retrieval. The resulting corpus of (original, rewritten) query pairs enables the training of rewriter models that are explicitly aware of document style and retrieval feedback. This work highlights a new direction in data-centric information retrieval, emphasizing how feedback loops and document-style alignment can enhance the reasoning and adaptability of RAG systems in real-world, domain-specific contexts.
Paper Structure (18 sections, 1 figure, 2 tables)

This paper contains 18 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Query Rewriting for Information Retrieval.
  4. Feedback-Driven Query Reformulation.
  5. Existing Query Rewriting Datasets.
  6. Methodology
  7. Initial Retrieval
  8. LLM-Guided Rewriting
  9. Verification via Re-Retrieval
  10. Dataset Assembly
  11. Experiments
  12. Setup
  13. Dataset Construction
  14. Qualitative Examples.
  15. Few-Shot Validation with Constructed Dataset
  16. ...and 3 more sections

Figures (1)

  • Figure 1: Overview of the retrieval feedback–driven dataset generation framework. Missed queries are rewritten via LLMs to match the style of the correct documents and validated through re-retrieval before being assembled into a style-aware corpus.