Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How Good Are LLMs at Processing Tool Outputs?

Kiran Kate, Yara Rizk, Poulami Ghosh, Ashu Gulati, Tathagata Chakraborti, Zidane Wright, Mayank Agarwal

TL;DR

This work analyzes how effectively LLMs process structured tool outputs in JSON form, a crucial step in tool-augmented automation. It creates a 1298-sample QA dataset spanning extractive, filtering, and aggregation tasks from multi-domain REST API responses and benchmarks 15 models with varied prompting strategies. The findings show JSON processing remains hard even for frontier models, but code-generation-based parsing and including a JSON schema significantly improve performance, especially for complex reasoning tasks, while full responses help when context allows. The study provides practical guidelines and a decision framework to optimize tool response processing in real-world systems and highlights directions for future end-to-end evaluation and handling of large, nested outputs.

Abstract

Most realistic task automation problems require large language models (LLMs) to call tools, which often return complex JSON responses. These responses must be further processed to derive the information necessary for task completion. The ability of LLMs to do so is under-studied. In this paper, we study the tool response processing task and LLMs' abilities to process structured (JSON) responses. We created a dataset for this task, and evaluated 15 open and closed weight models using multiple prompting approaches. Our results show that JSON processing remains a difficult task even for frontier models across multiple prompting strategies. The optimal response processing strategy depends on both the nature and size of the tool outputs, as well as the complexity of the required reasoning. Variations in processing approaches can lead to performance differences ranging from 3\% to 50\%.

How Good Are LLMs at Processing Tool Outputs?

TL;DR

This work analyzes how effectively LLMs process structured tool outputs in JSON form, a crucial step in tool-augmented automation. It creates a 1298-sample QA dataset spanning extractive, filtering, and aggregation tasks from multi-domain REST API responses and benchmarks 15 models with varied prompting strategies. The findings show JSON processing remains hard even for frontier models, but code-generation-based parsing and including a JSON schema significantly improve performance, especially for complex reasoning tasks, while full responses help when context allows. The study provides practical guidelines and a decision framework to optimize tool response processing in real-world systems and highlights directions for future end-to-end evaluation and handling of large, nested outputs.

Abstract

Most realistic task automation problems require large language models (LLMs) to call tools, which often return complex JSON responses. These responses must be further processed to derive the information necessary for task completion. The ability of LLMs to do so is under-studied. In this paper, we study the tool response processing task and LLMs' abilities to process structured (JSON) responses. We created a dataset for this task, and evaluated 15 open and closed weight models using multiple prompting approaches. Our results show that JSON processing remains a difficult task even for frontier models across multiple prompting strategies. The optimal response processing strategy depends on both the nature and size of the tool outputs, as well as the complexity of the required reasoning. Variations in processing approaches can lead to performance differences ranging from 3\% to 50\%.
Paper Structure (38 sections, 31 figures, 5 tables)

This paper contains 38 sections, 31 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Tool Response Processing
  3. Dataset Creation
  4. Common Approaches
  5. Experimental Setup
  6. Metrics
  7. Models
  8. Study Findings
  9. Impact of Response Length (RQ1)
  10. Impact of Answer Location (RQ2)
  11. Impact of Model Output Type (RQ3)
  12. Impact of Response Schema (RQ4)
  13. Classifying Queries
  14. Classifying Queries
  15. Impact of the JSON Response (RQ5)
  16. ...and 23 more sections

Figures (31)

  • Figure 1: A typical multi-turn tool use pipeline. Tool response processing is shown here as an explicit step to highlight its importance. This paper studies LLM-based approaches to tool response processing and measures the correctness of output "Data" from the processing step by creating a handcrafted evaluation dataset for it.
  • Figure 2: Tool response processing approaches
  • Figure 3: Exact match accuracy across models with increasing JSON size
  • Figure 4: Impact of response position
  • Figure 5: Performance comparison showing different metrics. Exact match is the strictest and LLM as a judge is the most lenient.
  • ...and 26 more figures