Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests

Jingjie Ning, João Coelho, Yibo Kong, Yunfan Long, Bruno Martins, João Magalhães, Jamie Callan, Chenyan Xiong

TL;DR

This study analyzes 14.44 million DRGym requests to understand how autonomous agents perform multi-step information seeking. It introduces a two-level labeling framework—session-level intents and trajectory-level reformulations—and a novel CTAR metric to quantify cross-step evidence grounding. Key findings include a drill-down bias with most sessions resolving within 10 steps, fixed retrieval depth across steps, and substantial cross-step term adoption from historical evidence, especially during specialization and exploration moves. The results offer design implications such as repetition-aware stopping, intent-adaptive retrieval budgeting, and explicit cross-step context management, with plans to release anonymized logs for reproducibility and further research.

Abstract

LLM-powered search agents are increasingly being used for multi-step information seeking tasks, yet the IR community lacks empirical understanding of how agentic search sessions unfold and how retrieved evidence is used. This paper presents a large-scale log analysis of agentic search based on 14.44M search requests (3.97M sessions) collected from DeepResearchGym, i.e. an open-source search API accessed by external agentic clients. We sessionize the logs, assign session-level intents and step-wise query-reformulation labels using LLM-based annotation, and propose Context-driven Term Adoption Rate (CTAR) to quantify whether newly introduced query terms are traceable to previously retrieved evidence. Our analyses reveal distinctive behavioral patterns. First, over 90% of multi-turn sessions contain at most ten steps, and 89% of inter-step intervals fall under one minute. Second, behavior varies by intent. Fact-seeking sessions exhibit high repetition that increases over time, while sessions requiring reasoning sustain broader exploration. Third, agents reuse evidence across steps. On average, 54% of newly introduced query terms appear in the accumulated evidence context, with contributions from earlier steps beyond the most recent retrieval. The findings suggest that agentic search may benefit from repetition-aware early stopping, intent-adaptive retrieval budgets, and explicit cross-step context tracking. We plan to release the anonymized logs to support future research.

Agentic Search in the Wild: Intents and Trajectory Dynamics from 14M+ Real Search Requests

TL;DR

This study analyzes 14.44 million DRGym requests to understand how autonomous agents perform multi-step information seeking. It introduces a two-level labeling framework—session-level intents and trajectory-level reformulations—and a novel CTAR metric to quantify cross-step evidence grounding. Key findings include a drill-down bias with most sessions resolving within 10 steps, fixed retrieval depth across steps, and substantial cross-step term adoption from historical evidence, especially during specialization and exploration moves. The results offer design implications such as repetition-aware stopping, intent-adaptive retrieval budgeting, and explicit cross-step context management, with plans to release anonymized logs for reproducibility and further research.

Abstract

LLM-powered search agents are increasingly being used for multi-step information seeking tasks, yet the IR community lacks empirical understanding of how agentic search sessions unfold and how retrieved evidence is used. This paper presents a large-scale log analysis of agentic search based on 14.44M search requests (3.97M sessions) collected from DeepResearchGym, i.e. an open-source search API accessed by external agentic clients. We sessionize the logs, assign session-level intents and step-wise query-reformulation labels using LLM-based annotation, and propose Context-driven Term Adoption Rate (CTAR) to quantify whether newly introduced query terms are traceable to previously retrieved evidence. Our analyses reveal distinctive behavioral patterns. First, over 90% of multi-turn sessions contain at most ten steps, and 89% of inter-step intervals fall under one minute. Second, behavior varies by intent. Fact-seeking sessions exhibit high repetition that increases over time, while sessions requiring reasoning sustain broader exploration. Third, agents reuse evidence across steps. On average, 54% of newly introduced query terms appear in the accumulated evidence context, with contributions from earlier steps beyond the most recent retrieval. The findings suggest that agentic search may benefit from repetition-aware early stopping, intent-adaptive retrieval budgets, and explicit cross-step context tracking. We plan to release the anonymized logs to support future research.
Paper Structure (48 sections, 3 equations, 8 figures, 9 tables)

This paper contains 48 sections, 3 equations, 8 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Human Search Behavior and Log Analysis:
  4. LLM Interaction Platforms and Usage Logs:
  5. Agentic Search Modeling, Benchmarks, and Infrastructures:
  6. Data and Log Processing
  7. DRGym Log Overview
  8. Scale and Coverage:
  9. Semantic Diversity:
  10. Query-Level Repetition:
  11. Log Preprocessing and Sessionization
  12. Methodology
  13. LLM-based Intent and Trajectory Labeling
  14. Session-Level Intent:
  15. Trajectory-Level Reformulation:
  16. ...and 33 more sections

Figures (8)

  • Figure 1: Intent--trajectory structure of agentic search logs.
  • Figure 2: Representativeness and diversity of the DRGym logs.
  • Figure 3: Left: distribution of session length (number of queries per session). Right: distribution of step-wise time intervals between consecutive requests.
  • Figure 4: Step-wise trajectory distribution trends for the first 10 steps across different task intents. Each sub-figure illustrates the evolving proportions of Specialization, Generalization, Exploration, and Repetition as the search session progresses.
  • Figure 5: A Declarative retry-loop example dominated by near-duplicate reformulations.
  • ...and 3 more figures