Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CASTER: Breaking the Cost-Performance Barrier in Multi-Agent Orchestration via Context-Aware Strategy for Task Efficient Routing

Shanyv Liu, Xuyang Yuan, Tao Chen, Zijun Zhan, Zhu Han, Danyang Zheng, Weishan Zhang, Shaohua Cao

TL;DR

CASTER addresses the cost-performance paradox in graph-based multi-agent systems by introducing a context-aware router that combines semantic task content with structural context to dynamically allocate sub-tasks to appropriate model backends. It employs a Dual-Branch Feature Fusion Network to predict the need for a strong vs. weak model and is integrated as a per-step interceptor in LangGraph, enabling on-demand routing with rollback. The training pipeline combines a Cold Start pre-training phase with an Iterative Evolution phase using on-policy negative feedback, achieving up to 72.4% inference cost reduction while maintaining or improving task success across Software, Data, Science, and Security domains, outperforming FrugalGPT and heuristic routing. The approach is domain-agnostic and promotes greener AI by reducing compute and latency, with potential for broader applications beyond the current four domains.

Abstract

Graph-based Multi-Agent Systems (MAS) enable complex cyclic workflows but suffer from inefficient static model allocation, where deploying strong models uniformly wastes computation on trivial sub-tasks. We propose CASTER (Context-Aware Strategy for Task Efficient Routing), a lightweight router for dynamic model selection in graph-based MAS. CASTER employs a Dual-Signal Router that combines semantic embeddings with structural meta-features to estimate task difficulty. During training, the router self-optimizes through a Cold Start to Iterative Evolution paradigm, learning from its own routing failures via on-policy negative feedback. Experiments using LLM-as-a-Judge evaluation across Software Engineering, Data Analysis, Scientific Discovery, and Cybersecurity demonstrate that CASTER reduces inference cost by up to 72.4% compared to strong-model baselines while matching their success rates, and consistently outperforms both heuristic routing and FrugalGPT across all domains.

CASTER: Breaking the Cost-Performance Barrier in Multi-Agent Orchestration via Context-Aware Strategy for Task Efficient Routing

TL;DR

CASTER addresses the cost-performance paradox in graph-based multi-agent systems by introducing a context-aware router that combines semantic task content with structural context to dynamically allocate sub-tasks to appropriate model backends. It employs a Dual-Branch Feature Fusion Network to predict the need for a strong vs. weak model and is integrated as a per-step interceptor in LangGraph, enabling on-demand routing with rollback. The training pipeline combines a Cold Start pre-training phase with an Iterative Evolution phase using on-policy negative feedback, achieving up to 72.4% inference cost reduction while maintaining or improving task success across Software, Data, Science, and Security domains, outperforming FrugalGPT and heuristic routing. The approach is domain-agnostic and promotes greener AI by reducing compute and latency, with potential for broader applications beyond the current four domains.

Abstract

Graph-based Multi-Agent Systems (MAS) enable complex cyclic workflows but suffer from inefficient static model allocation, where deploying strong models uniformly wastes computation on trivial sub-tasks. We propose CASTER (Context-Aware Strategy for Task Efficient Routing), a lightweight router for dynamic model selection in graph-based MAS. CASTER employs a Dual-Signal Router that combines semantic embeddings with structural meta-features to estimate task difficulty. During training, the router self-optimizes through a Cold Start to Iterative Evolution paradigm, learning from its own routing failures via on-policy negative feedback. Experiments using LLM-as-a-Judge evaluation across Software Engineering, Data Analysis, Scientific Discovery, and Cybersecurity demonstrate that CASTER reduces inference cost by up to 72.4% compared to strong-model baselines while matching their success rates, and consistently outperforms both heuristic routing and FrugalGPT across all domains.
Paper Structure (62 sections, 2 equations, 22 figures, 18 tables, 3 algorithms)

This paper contains 62 sections, 2 equations, 22 figures, 18 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. System Architecture
  5. CASTER Design
  6. CASTER Architecture: Dual-Branch Feature Fusion Network
  7. Training Strategy
  8. Pre-training and Cold Start Strategy
  9. Iterative Fine-tuning via Negative Feedback
  10. Results & Analysis
  11. Experiments Setup
  12. Model Evaluation
  13. Cost Evaluation
  14. Cumulative Cost and Average Cost
  15. Cost Distribution
  16. ...and 47 more sections

Figures (22)

  • Figure 1: The overall architecture of the CASTER framework. The system begins with mock data and dynamic task generation via GPT-4o. The core Router integrates semantic and meta-features to dispatch tasks, evolving through cold start and on-policy negative feedback mechanisms. Tasks are executed by domain-specific agents (Software, Data, Science, Security) and evaluated against a comprehensive benchmark to ensure multi-model capability.
  • Figure 2: CASTER Confidence Validation. Inference scores across Software, Data, Science, and Security. The threshold ($y=0.5$, dashed) separates simple tasks (blue, Weak Model) from complex ones (yellow, Strong Model). Results confirm the router's efficacy in distinguishing task complexity.
  • Figure 3: Overview of the Domain-Specific Multi-Agent Workflows.
  • Figure 4: Three-Stage Hybrid Training Strategy
  • Figure 5: Dynamic Adversarial Task Generation
  • ...and 17 more figures