Natural Language Tools: A Natural Language Approach to Tool Calling In Large Language Agents

TL;DR

The paper addresses bottlenecks in structured tool calling by introducing Natural Language Tools (NLT), which decouples tool selection from response generation and replaces JSON with natural-language outputs. Its three-stage architecture (tool selection, tool execution, response) enables parallel multi-tool calls guided by a purpose-built selector prompt, reducing token overhead and task interference. Across 10 frontier models and 6,400 trials in customer-service and mental-health domains, NLT achieves an 18.4 percentage point accuracy gain (69.1% to 87.5%), with open-weight models benefiting most and robustness to prompt perturbations maintained. The work highlights mechanisms for improvement, discusses training and evaluation implications, and candidly addresses latency, complexity, and cross-model applicability, suggesting substantial practical impact for agentic systems, especially those lacking native tool-calling support.

Abstract

We present Natural Language Tools (NLT), a framework that replaces programmatic JSON tool calling in large language models (LLMs) with natural language outputs. By decoupling tool selection from response generation, NLT eliminates task interference and format constraints that degrade tool call performance. When evaluated across 10 models and 6,400 trials spanning customer service and mental health domains, NLT improves tool calling accuracy by 18.4 percentage points while reducing output variance by 70%. Open-weight models see the largest gains, surpassing flagship closed-weight alternatives, with implications for model training in both reinforcement learning and supervised fine-tuning stages. These improvements persist under prompt perturbations and extend tool-calling capabilities to models lacking native support.

Figures (7)

• Figure 1: An example of a JSON-formatted tool call for a "talk to a human" request within a customer service scenario.
• Figure 2: The above represents a simplified NLT architecture with a single input LLM and a single output LLM. As NLT is fully modular, these layers can be abstracted to include any level of additional complexity. For example, one tool may be chained into another, or the final output model may simply be the next step in an agentic process.
• Figure 3: Side by side comparisons between NLT and Structured approaches, across three categories: overall results, closed-weight models, and open-weight models.
• Figure 4: Relative comparisons between Structured and NLT tool-calling approaches across core model set. Mean tool-call accuracy is averaged across domains and perturbations.
• Figure 5: Side by side comparisons of model performance shifts between structured tool calls and NLT, with variance represented by the y-axis and mean accuracy by the x-axis. Accuracy and variance are calculated across the aggregate of Alex and Sage scenarios, as well as perturbed and non-perturbed prompts. Full per-model results are reported in Appendix B.