Dynamic Tool Dependency Retrieval for Efficient Function Calling

TL;DR

Dynamic Tool Dependency Retrieval (DTDR) tackles the challenge of selecting relevant tools for on-device LLM function calling by conditioning tool retrieval on both the user query and the evolving plan history. It introduces two lightweight variants, DTDR-C (clustering-based) and DTDR-L (learned linear), to infer a minimal, task-specific dependency subgraph and inject it into prompts. Across multiple datasets and model backbones, DTDR improves retrieval precision, function calling accuracy, and end-to-end success rates while reducing prompt length, demonstrating strong practical impact for memory- and latency-constrained agents. The work lays a foundation for robust on-device tool use and invites extensions to multimodal tools and evolving tool ecosystems.

Abstract

Function calling agents powered by Large Language Models (LLMs) select external tools to automate complex tasks. On-device agents typically use a retrieval module to select relevant tools, improving performance and reducing context length. However, existing retrieval methods rely on static and limited inputs, failing to capture multi-step tool dependencies and evolving task context. This limitation often introduces irrelevant tools that mislead the agent, degrading efficiency and accuracy. We propose Dynamic Tool Dependency Retrieval (DTDR), a lightweight retrieval method that conditions on both the initial query and the evolving execution context. DTDR models tool dependencies from function calling demonstrations, enabling adaptive retrieval as plans unfold. We benchmark DTDR against state-of-the-art retrieval methods across multiple datasets and LLM backbones, evaluating retrieval precision, downstream task accuracy, and computational efficiency. Additionally, we explore strategies to integrate retrieved tools into prompts. Our results show that dynamic tool retrieval improves function calling success rates between $23\%$ and $104\%$ compared to state-of-the-art static retrievers.

Figures (5)

• Figure 1: Dynamic Tool Dependency Retrieval. Given demonstration data for a set of tools, previous work retrieve tools based on either a) the natural language query (highlighted in orange) or b) the latest executed tool call in the plan (the red triangle, highlighted in teal). We instead propose a retrieval method which is dynamically conditioned on both the query and the current history of tool calls (blue square, then red triangle). This dynamic method allows to retrieve only tools that are strictly relevant to both query (orange ) and tool context (teal ).
• Figure 2: System diagram for DTDR. On the left, the user query and tool history are input to DTDR to retrieve the most likely next tools. The LLM $\pi$ selects the next tool among this set. On the right, we show the two alternative instantiations for the retriever: a) DTDR-C, based on a clustering step to retrieve an explicit graph of tool dependencies; and b) DTDR-L, based on a learned linear classifier implicitly modeling tool dependencies. Both systems are conditioned on user query and full tool history.
• Figure 3: Comparison of efficient ICL methods against ICL with Raw Demonstrations and the baseline without ICL. All results are conditioned using the DTDR-C retriever. Pruning irrelevant tools from the prompt has a greater impact for smaller models that cannot handle longer contexts.
• Figure 4: Prompt length across different methods and datasets. Our method reduces the prompt length by: 1) efficiently encoding ICL examples as tool dependencies instead of Raw Demonstrations, and 2) only retrieving tool dependencies which are relevant for the test query.
• Figure 5: Ablations on: a) history length, b) # of k-means clusters, and c) # of demonstrations.