Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AuTAgent: A Reinforcement Learning Framework for Tool-Augmented Audio Reasoning

Siqian Tong, Xuan Li, Yiwei Wang, Baolong Bi, Yujun Cai, Shenghua Liu, Yuchen He, Chengpeng Hao

TL;DR

A reinforcement learning framework that learns to filter out irrelevant tools and invokes external assistance only when it yields a net performance gain over the base model, and highlights the complementary role of external tools in augmenting audio model reasoning.

Abstract

Large Audio Language Models (LALMs) excel at perception but struggle with complex reasoning requiring precise acoustic measurements. While external tools can extract fine-grained features like exact tempo or pitch, effective integration remains challenging: naively using all tools causes information overload, while prompt-based selection fails to assess context-dependent utility. To address this, we propose AuTAgent (Audio Tool Agent), a reinforcement learning framework that learns when and which tools to invoke. By employing a sparse-feedback training strategy with a novel Differential Reward mechanism, the agent learns to filter out irrelevant tools and invokes external assistance only when it yields a net performance gain over the base model. Experimental results confirm that AuTAgent complements the representation bottleneck of LALMs by providing verifiable acoustic evidence. It improves accuracy by 4.20% / 6.20% and 9.80% / 8.00% for open-source and closed-source backbones on the MMAU Test-mini and the MMAR benchmarks, respectively. In addition, further experiments demonstrate exceptional transferability. We highlight the complementary role of external tools in augmenting audio model reasoning.

AuTAgent: A Reinforcement Learning Framework for Tool-Augmented Audio Reasoning

TL;DR

A reinforcement learning framework that learns to filter out irrelevant tools and invokes external assistance only when it yields a net performance gain over the base model, and highlights the complementary role of external tools in augmenting audio model reasoning.

Abstract

Large Audio Language Models (LALMs) excel at perception but struggle with complex reasoning requiring precise acoustic measurements. While external tools can extract fine-grained features like exact tempo or pitch, effective integration remains challenging: naively using all tools causes information overload, while prompt-based selection fails to assess context-dependent utility. To address this, we propose AuTAgent (Audio Tool Agent), a reinforcement learning framework that learns when and which tools to invoke. By employing a sparse-feedback training strategy with a novel Differential Reward mechanism, the agent learns to filter out irrelevant tools and invokes external assistance only when it yields a net performance gain over the base model. Experimental results confirm that AuTAgent complements the representation bottleneck of LALMs by providing verifiable acoustic evidence. It improves accuracy by 4.20% / 6.20% and 9.80% / 8.00% for open-source and closed-source backbones on the MMAU Test-mini and the MMAR benchmarks, respectively. In addition, further experiments demonstrate exceptional transferability. We highlight the complementary role of external tools in augmenting audio model reasoning.
Paper Structure (32 sections, 3 equations, 4 figures, 5 tables, 1 algorithm)

This paper contains 32 sections, 3 equations, 4 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminary Analysis: Potential and Pitfalls of Tool Augmentation
  3. Method
  4. Problem Definition
  5. Policy Optimization via Differential Reward Mechanism
  6. Differential Reward Formulation.
  7. Group Relative Update.
  8. Experimental Setup
  9. Dataset Construction
  10. Implementation Details
  11. Evaluation Benchmarks
  12. Modular Tool Library
  13. Results and Analysis
  14. Main Results
  15. Transferability Analysis
  16. ...and 17 more sections

Figures (4)

  • Figure 1: Challenges in LALMs and our solution. AuTAgent employs a dynamic tool-selection policy to extract precise evidence (e.g., exact BPM), ensuring accurate and efficient audio reasoning.
  • Figure 2: Comparison of MMAU Test-mini accuracy across single tools (T1–T6), the All-Tools baseline, the No-Tool baseline and a theoretical upper bound (Upper).
  • Figure 3: Training framework of AuTAgent. Given an audio-query pair, the Audio Tool Agent select specific tools and generate structured audio evidence, which augments the input for a frozen backend Reasoner. By introducing a Baseline-Subtracted Differential Reward mechanism, the agent receives a positive reward only when it successfully corrects baseline errors. This constraint drives the discovery of effective, context-aware tool combinations through GRPO policy updates, effectively suppressing redundancy.
  • Figure 4: Distribution Shift in Tool Invocation frequency across the description-based agent and our RL-trained agent.