On Generalization in Agentic Tool Calling: CoreThink Agentic Reasoner and MAVEN Dataset

TL;DR

The paper addresses the generalization gap in agentic tool calling by introducing MAVEN, an adversarial, out-of-distribution benchmark for long-horizon, tool-augmented reasoning, and the CoreThink Agentic Reasoner, a neuro-symbolic augmentation that decomposes tasks, orchestrates tools, and verifies intermediate results. CoreThink achieves state-of-the-art performance across standard benchmarks with substantial efficiency gains, while MAVEN reveals robustness gaps in many frontier models. The authors demonstrate that combining structured symbolic reasoning with LLMs improves generalization and interpretability, and they release MAVEN and evaluation scripts to foster reproducible, adversarially challenging research in scientific reasoning agents. Overall, the work advances both methodology and benchmarks for resilient, verifiable agentic systems with practical implications for deployment in diverse domains.

Abstract

Generalization across Agentic tool-calling environments remains a key unsolved challenge in developing reliable agentic reasoning systems. While large language models (LLMs) demonstrate strong performance on isolated benchmarks, their ability to transfer reasoning strategies and co-ordinate tools across diverse domains is poorly understood. In this work, we conduct a large-scale evaluation of state-of-the-art LLMs on multiple tool-calling benchmarksBFCL v3, TauBench, Tau2Bench, and AceBenchand introduce MAVEN (Math & Physics Adversarial Verification & Evaluation Network), a new out of distribution (OOD) benchmark designed to stress-test multi-step reasoning through explicit verification and adversarial task composition. Our results show that most current models achieve below 50% accuracy on MAVEN, revealing a significant generalization gap across tool-use settings. To address this, we present the CoreThink Agentic Reasoner, a framework that augments LLMs with a lightweight symbolic reasoning layer for structured decomposition and adaptive tool orchestration. Without additional training, it generalizes across all benchmarks, achieving state-of-the-art performance with 530% improvements over existing baselines at roughly one-tenth the computational cost.

Figures (6)

• Figure 1: The system processes conversational input through three stages: Context Buffering extracts and structures relevant information, Action Synthesis generates atomic, testable tasks while handling early termination and missing prerequisites, and Invocation Generation produces machine-interpretable actions with auditability, keeping reasoning and execution separated.
• Figure 2: Schematic of the MAVEN evaluation setup. A user supplies a multi-step math or physics problem; the Agent orchestrates calls to external tools (e.g., solve_equation, integrate, matrix_determinant, linear_regression), verifies intermediate results at each step, and aggregates those results to produce the final solution. Right: an example MAVEN trajectory showing sequential, step-wise tool calls with intermediate verification and final answer aggregation.
• Figure 3: Comparison of CoreThink AI and GPT-OSS-120b scores across tool-calling benchmarks. CoreThink consistently outperforms the base LLM, demonstrating the benefits of its NeuroSymbolic reasoning layer.
• Figure 4: Example (schematic)
• Figure 5: Minimal MCP interaction example (schematic)