Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GenCRF: Generative Clustering and Reformulation Framework for Enhanced Intent-Driven Information Retrieval

Wonduk Seo, Haojie Zhang, Yueyang Zhang, Changhao Zhang, Songyao Duan, Lixin Su, Daiting Shi, Jiashu Zhao, Dawei Yin

TL;DR

GenCRF is proposed, a Generative Clustering and Reformulation Framework to capture diverse intentions adaptively based on multiple differentiated, well-generated queries in the retrieval phase for the first time and achieves state-of-the-art performance.

Abstract

Query reformulation is a well-known problem in Information Retrieval (IR) aimed at enhancing single search successful completion rate by automatically modifying user's input query. Recent methods leverage Large Language Models (LLMs) to improve query reformulation, but often generate limited and redundant expansions, potentially constraining their effectiveness in capturing diverse intents. In this paper, we propose GenCRF: a Generative Clustering and Reformulation Framework to capture diverse intentions adaptively based on multiple differentiated, well-generated queries in the retrieval phase for the first time. GenCRF leverages LLMs to generate variable queries from the initial query using customized prompts, then clusters them into groups to distinctly represent diverse intents. Furthermore, the framework explores to combine diverse intents query with innovative weighted aggregation strategies to optimize retrieval performance and crucially integrates a novel Query Evaluation Rewarding Model (QERM) to refine the process through feedback loops. Empirical experiments on the BEIR benchmark demonstrate that GenCRF achieves state-of-the-art performance, surpassing previous query reformulation SOTAs by up to 12% on nDCG@10. These techniques can be adapted to various LLMs, significantly boosting retriever performance and advancing the field of Information Retrieval.

GenCRF: Generative Clustering and Reformulation Framework for Enhanced Intent-Driven Information Retrieval

TL;DR

GenCRF is proposed, a Generative Clustering and Reformulation Framework to capture diverse intentions adaptively based on multiple differentiated, well-generated queries in the retrieval phase for the first time and achieves state-of-the-art performance.

Abstract

Query reformulation is a well-known problem in Information Retrieval (IR) aimed at enhancing single search successful completion rate by automatically modifying user's input query. Recent methods leverage Large Language Models (LLMs) to improve query reformulation, but often generate limited and redundant expansions, potentially constraining their effectiveness in capturing diverse intents. In this paper, we propose GenCRF: a Generative Clustering and Reformulation Framework to capture diverse intentions adaptively based on multiple differentiated, well-generated queries in the retrieval phase for the first time. GenCRF leverages LLMs to generate variable queries from the initial query using customized prompts, then clusters them into groups to distinctly represent diverse intents. Furthermore, the framework explores to combine diverse intents query with innovative weighted aggregation strategies to optimize retrieval performance and crucially integrates a novel Query Evaluation Rewarding Model (QERM) to refine the process through feedback loops. Empirical experiments on the BEIR benchmark demonstrate that GenCRF achieves state-of-the-art performance, surpassing previous query reformulation SOTAs by up to 12% on nDCG@10. These techniques can be adapted to various LLMs, significantly boosting retriever performance and advancing the field of Information Retrieval.
Paper Structure (39 sections, 8 equations, 9 figures, 13 tables, 1 algorithm)

This paper contains 39 sections, 8 equations, 9 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. METHODOLOGY
  4. Overview of GenCRF
  5. Generation and Clustering
  6. Weighted Aggregation Strategies
  7. Similarity Dynamic Weights (SimDW).
  8. Score Dynamic Weights (ScoreDW).
  9. Fine-Tuning for ScoreDW.
  10. Query Evaluation Rewarding Model
  11. EXPERIMENTS
  12. Setup
  13. Experimental Datasets
  14. Models Used
  15. LLMs:
  16. ...and 24 more sections

Figures (9)

  • Figure 1: Overview of the GenCRF: Generative Clustering and Reformulation Framework
  • Figure 2: Initial Weight Comparison of FW and DW
  • Figure 3: nDCG@10 scores for different prompt quantities
  • Figure 4: Performance comparison of GenCRF with GenQR using DC and FW strategies
  • Figure 5: Comparsion of nDCG@10 scores between GenCRF's prompts and baseline methods
  • ...and 4 more figures