Q-PEFT: Query-dependent Parameter Efficient Fine-tuning for Text Reranking with Large Language Models

TL;DR

Q-PEFT introduces a query-dependent, parameter-efficient fine-tuning framework for text reranking that freezes the LLM and learns a lightweight QD module. It presents two variants: Q-PEFT-R, which uses top-$k$ tokens from documents, and Q-PEFT-A, which uses a differentiable multi-head attention mechanism to model query-document relevance; both produce document-specific synthetic queries to improve ranking via log-likelihood scoring $I_{\theta,\Phi}(q|d,s)$. Extensive experiments across four public datasets with diverse retrievers show consistent gains over strong baselines, with larger gains on unsupervised retrievers and robust behavior across multiple LLMs, though some MOE models may underperform depending on architecture. The results demonstrate the practicality of end-to-end, query-aware PEFT for IR tasks and highlight future work combining QD modules with soft prompts for broader adaptability. $L(q|d,s)$ and $I_{\theta,\Phi}$ are central scoring components, enabling effective optimization while maintaining the LLM’s foundational capabilities.

Abstract

Parameter Efficient Fine-Tuning (PEFT) methods have been extensively utilized in Large Language Models (LLMs) to improve the down-streaming tasks without the cost of fine-tuing the whole LLMs. Recent studies have shown how to effectively use PEFT for fine-tuning LLMs in ranking tasks with convincing performance; there are some limitations, including the learned prompt being fixed for different documents, overfitting to specific tasks, and low adaptation ability. In this paper, we introduce a query-dependent parameter efficient fine-tuning (Q-PEFT) approach for text reranking to leak the information of the true queries to LLMs and then make the generation of true queries from input documents much easier. Specifically, we utilize the query to extract the top-$k$ tokens from concatenated documents, serving as contextual clues. We further augment Q-PEFT by substituting the retrieval mechanism with a multi-head attention layer to achieve end-to-end training and cover all the tokens in the documents, guiding the LLMs to generate more document-specific synthetic queries, thereby further improving the reranking performance. Extensive experiments are conducted on four public datasets, demonstrating the effectiveness of our proposed approach.

Figures (4)

• Figure 1: The architecture of the proposed Q-PEFT. For each instance $\left\langle q_c, d_c^{+}, d_{c}^{-}\right\rangle$, we compute pointwise loss $L_{point}$ and pairwise loss $L_{pair}$ and add them together as final loss $L$. During training, LLM's original parameters $\Phi$ are fixed, and only the parameters of Query-dependent module $\theta$ are updated.
• Figure 2: Performance comparison of different hard prompts on various Datasets, the performance of Q-PEFT evaluated by the average Recall@10 Metric across various retrievers.
• Figure 3: Comparative analysis of Recall@10 on NQ and Webq dataset for UPR and Q-PEFT-based models across various retrievers and different training sizes ($k$).
• Figure 4: Comparative analysis of the average Recall@10 for five retrievers using UPR and Q-PEFT-based models across various training sizes ($k$), evaluated respectively on the NQ, Webq, Trivia, and Squad1 datasets.