LLM-VPRF: Large Language Model Based Vector Pseudo Relevance Feedback

TL;DR

This work demonstrates that Vector Pseudo Relevance Feedback (VPRF) can extend to Large Language Model (LLM)–based dense retrievers, enabling iterative refinement of query embeddings via feedback passages. It formalizes two VPRF variants, Average and Rocchio, for LLM embeddings and evaluates across BEIR and TREC Deep Learning benchmarks using three state-of-the-art LLM backbones (PromptReps, RepLLaMa, LLM2Vec). The results show consistent performance gains with modest computational overhead, though the magnitude and stability of improvements depend on the specific LLM architecture and dataset characteristics. The findings bridge VPRF with modern LLM-based retrieval, offering practical guidance for deploying feedback-enhanced dense retrieval in real-world systems while highlighting areas for improved feedback selection and broader model coverage.

Abstract

Vector Pseudo Relevance Feedback (VPRF) has shown promising results in improving BERT-based dense retrieval systems through iterative refinement of query representations. This paper investigates the generalizability of VPRF to Large Language Model (LLM) based dense retrievers. We introduce LLM-VPRF and evaluate its effectiveness across multiple benchmark datasets, analyzing how different LLMs impact the feedback mechanism. Our results demonstrate that VPRF's benefits successfully extend to LLM architectures, establishing it as a robust technique for enhancing dense retrieval performance regardless of the underlying models. This work bridges the gap between VPRF with traditional BERT-based dense retrievers and modern LLMs, while providing insights into their future directions.