Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Compute Allocation for Reasoning-Intensive Retrieval Agents

Sreeja Apparaju, Nilesh Gupta

Abstract

As agents operate over long horizons, their memory stores grow continuously, making retrieval critical to accessing relevant information. Many agent queries require reasoning-intensive retrieval, where the connection between query and relevant documents is implicit and requires inference to bridge. LLM-augmented pipelines address this through query expansion and candidate re-ranking, but introduce significant inference costs. We study computation allocation in reasoning-intensive retrieval pipelines using the BRIGHT benchmark and Gemini 2.5 model family. We vary model capacity, inference-time thinking, and re-ranking depth across query expansion and re-ranking stages. We find that re-ranking benefits substantially from stronger models (+7.5 NDCG@10) and deeper candidate pools (+21% from $k$=10 to 100), while query expansion shows diminishing returns beyond lightweight models (+1.1 NDCG@10 from weak to strong). Inference-time thinking provides minimal improvement at either stage. These results suggest that compute should be concentrated on re-ranking rather than distributed uniformly across pipeline stages.

Compute Allocation for Reasoning-Intensive Retrieval Agents

Abstract

As agents operate over long horizons, their memory stores grow continuously, making retrieval critical to accessing relevant information. Many agent queries require reasoning-intensive retrieval, where the connection between query and relevant documents is implicit and requires inference to bridge. LLM-augmented pipelines address this through query expansion and candidate re-ranking, but introduce significant inference costs. We study computation allocation in reasoning-intensive retrieval pipelines using the BRIGHT benchmark and Gemini 2.5 model family. We vary model capacity, inference-time thinking, and re-ranking depth across query expansion and re-ranking stages. We find that re-ranking benefits substantially from stronger models (+7.5 NDCG@10) and deeper candidate pools (+21% from =10 to 100), while query expansion shows diminishing returns beyond lightweight models (+1.1 NDCG@10 from weak to strong). Inference-time thinking provides minimal improvement at either stage. These results suggest that compute should be concentrated on re-ranking rather than distributed uniformly across pipeline stages.
Paper Structure (10 sections, 2 figures, 2 tables)

This paper contains 10 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Pipeline Architecture
  4. Results
  5. Scaling in Query Expansion (QE)
  6. Scaling in Re-ranking (RR)
  7. Ablation I: Varying Re-ranking Model Variant
  8. Ablation II: Varying Re-ranking Depth
  9. Conclusion
  10. Future Work:

Figures (2)

  • Figure 1: Example of simplified LLM-augmented retrieval pipeline for reasoning-intensive queries
  • Figure 2: Impact of re-ranking depth across query expansion strategies. Increasing the candidate pool from $k=10$ to $k=100$ consistently improves NDCG@10 by approximately 20% across all query expansion models, demonstrating that episodic retrieval depth is a high-leverage compute allocation mechanism.