LightSearcher: Efficient DeepSearch via Experiential Memory

TL;DR

Deep reasoning models rely on external knowledge via DeepSearch but face a persistent accuracy–efficiency trade-off when using RL-controlled tool invocations. LightSearcher introduces contrastive experiential memory to distill explicit, interpretable guidance from successful reasoning trajectories and an adaptive reward shaping mechanism that penalizes redundant tool calls only when answers are correct. The approach, trained with GRPO and enhanced prompts, substantially reduces tool usage and inference time while preserving state-of-the-art or competitive accuracy across four multi-hop QA benchmarks, including strong out-of-domain generalization. These results demonstrate a practical path to more efficient, reliable, tool-augmented reasoning with potential applicability beyond QA to other knowledge-intensive tasks.

Abstract

DeepSearch paradigms have become a core enabler for deep reasoning models, allowing them to invoke external search tools to access up-to-date, domain-specific knowledge beyond parametric boundaries, thereby enhancing the depth and factual reliability of reasoning. Building upon this foundation, recent advances in reinforcement learning (RL) have further empowered models to autonomously and strategically control search tool usage, optimizing when and how to query external knowledge sources. Yet, these RL-driven DeepSearch systems often reveal a see-saw trade-off between accuracy and efficiency-frequent tool invocations can improve factual correctness but lead to unnecessary computational overhead and diminished efficiency. To address this challenge, we propose LightSearcher, an efficient RL framework that incorporates textual experiential memory by learning contrastive reasoning trajectories to generate interpretable summaries of successful reasoning patterns. In addition, it employs an adaptive reward shaping mechanism that penalizes redundant tool calls only in correct-answer scenarios. This design effectively balances the inherent accuracy-efficiency trade-off in DeepSearch paradigms. Experiments on four multi-hop QA benchmarks show that LightSearcher maintains accuracy comparable to SOTA baseline ReSearch, while reducing search tool invocations by 39.6%, inference time by 48.6%, and token consumption by 21.2%, demonstrating its superior efficiency.

Figures (6)

• Figure 1: Illustration of Excessive Search Tool Usage in existing DeepSearch systems across both Easy and Hard queries, leading to degraded efficiency in deep reasoning models.
• Figure 2: Overview of the LightSearcher framework pipeline. It contains three core modules: Contrastive Experience Reasoning, Adaptive Reward Shaping, and Experience-based RL training modules.
• Figure 3: Performance comparison of LightSearcher and baseline methods across four datasets on Qwen2.5-7B-Instruct. The figure presents the relationship between F1 score, LLM judge, and search tool calls, demonstrating that our model achieves the optimal balance in terms of effectiveness and search overhead.
• Figure 4: The impact of experience incorporation in inference on F1 score and Tool calls across training steps.
• Figure 5: Computational efficiency comparison between LightSearcher and ReSearch, evaluating Inference Time and Token Consumption.