DLLM-Searcher: Adapting Diffusion Large Language Model for Search Agents

TL;DR

DLLM-Searcher is proposed, an optimization framework for dLLM-based Search Agents that designs a two-stage post-training pipeline encompassing Agentic Supervised Fine-Tuning (Agentic SFT) and Agentic Variance-Reduced Preference Optimization Agentic VRPO, which enhances the backbone dLLM's information seeking and reasoning capabilities.

Abstract

Recently, Diffusion Large Language Models (dLLMs) have demonstrated unique efficiency advantages, enabled by their inherently parallel decoding mechanism and flexible generation paradigm. Meanwhile, despite the rapid advancement of Search Agents, their practical deployment is constrained by a fundamental limitation, termed as 1) Latency Challenge: the serial execution of multi-round reasoning, tool calling, and tool response waiting under the ReAct agent paradigm induces severe end-to-end latency. Intuitively, dLLMs can leverage their distinctive strengths to optimize the operational efficiency of agents under the ReAct agent paradigm. Practically, existing dLLM backbones face the 2) Agent Ability Challenge. That is, existing dLLMs exhibit remarkably weak reasoning and tool-calling capabilities, preventing these advantages from being effectively realized in practice. In this paper, we propose DLLM-Searcher, an optimization framework for dLLM-based Search Agents. To solve the Agent Ability Challenge, we design a two-stage post-training pipeline encompassing Agentic Supervised Fine-Tuning (Agentic SFT) and Agentic Variance-Reduced Preference Optimization Agentic VRPO, which enhances the backbone dLLM's information seeking and reasoning capabilities. To mitigate the Latency Challenge, we leverage the flexible generation mechanism of dLLMs and propose a novel agent paradigm termed Parallel-Reasoning and Acting P-ReAct. P-ReAct guides the model to prioritize decoding tool_call instructions, thereby allowing the model to keep thinking while waiting for the tool's return. Experimental results demonstrate that DLLM-Searcher achieves performance comparable to mainstream LLM-based search agents and P-ReAct delivers approximately 15% inference acceleration. Our code is available at https://anonymous.4open.science/r/DLLM-Searcher-553C

Figures (4)

• Figure 1: LLM-based Search Agent (ReAct) vs. dLLM-based Search Agent (ReAct and P-ReAct). Top & Middle: Standard ReAct paradigm suffers from high latency due to serial thinking and tool-calling regions generation and tool response waiting, while vanilla dLLMs fail due to tool-calling format errors. Bottom: P-ReAct prioritize tool-calling generation, enabling the model to keep thinking during tool execution.
• Figure 2: DLLM-Searcher includes training process and inference process. In training, both Agentic SFT and Agentic VRPO use Block Attention and Agentic Noising to compute the Agentic ELBO, which serves to estimate $\log\pi_{\theta}(y \mid x)$. In inference, we employ the P-ReAct agent paradigm . We pre-fill special boundary tokens and apply an additional confidence bias to encourage the model to decode the tool_call region with priority.
• Figure 3: Accuracy and average end-to-end latency comparison of DLLM-Searcher under P-ReAct and ReAct paradigms on multi-hop QA tasks.
• Figure 4: Comparison of accuracy changes on multi-hop QA tasks between DLLM-Searcher and Qwen3 series models when switching from ReAct to P-ReAct.