Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EvoRoute: Experience-Driven Self-Routing LLM Agent Systems

Guibin Zhang, Haiyang Yu, Kaiming Yang, Bingli Wu, Fei Huang, Yongbin Li, Shuicheng Yan

TL;DR

EvoRoute introduces an experience-driven, self-evolving routing framework that dynamically selects Pareto-optimal LLM backbones for each sub-task within complex agentic workflows. By building a continually growing knowledge base, performing multi-faceted retrieval, applying Pareto filtering, and using Thompson sampling for robust model choice, EvoRoute simultaneously improves task performance while dramatically reducing cost and latency. The approach is validated on challenging benchmarks like GAIA and BrowseComp+ and shows clear improvements over static mappings and existing routing baselines, addressing the inherent trade-offs among performance, efficiency, and economy. This work offers a scalable, practical pathway to deploying powerful agentic AI systems in real-world settings with favorable cost-to-performance ratios and faster task completion times.

Abstract

Complex agentic AI systems, powered by a coordinated ensemble of Large Language Models (LLMs), tool and memory modules, have demonstrated remarkable capabilities on intricate, multi-turn tasks. However, this success is shadowed by prohibitive economic costs and severe latency, exposing a critical, yet underexplored, trade-off. We formalize this challenge as the \textbf{Agent System Trilemma}: the inherent tension among achieving state-of-the-art performance, minimizing monetary cost, and ensuring rapid task completion. To dismantle this trilemma, we introduce EvoRoute, a self-evolving model routing paradigm that transcends static, pre-defined model assignments. Leveraging an ever-expanding knowledge base of prior experience, EvoRoute dynamically selects Pareto-optimal LLM backbones at each step, balancing accuracy, efficiency, and resource use, while continually refining its own selection policy through environment feedback. Experiments on challenging agentic benchmarks such as GAIA and BrowseComp+ demonstrate that EvoRoute, when integrated into off-the-shelf agentic systems, not only sustains or enhances system performance but also reduces execution cost by up to $80\%$ and latency by over $70\%$.

EvoRoute: Experience-Driven Self-Routing LLM Agent Systems

TL;DR

EvoRoute introduces an experience-driven, self-evolving routing framework that dynamically selects Pareto-optimal LLM backbones for each sub-task within complex agentic workflows. By building a continually growing knowledge base, performing multi-faceted retrieval, applying Pareto filtering, and using Thompson sampling for robust model choice, EvoRoute simultaneously improves task performance while dramatically reducing cost and latency. The approach is validated on challenging benchmarks like GAIA and BrowseComp+ and shows clear improvements over static mappings and existing routing baselines, addressing the inherent trade-offs among performance, efficiency, and economy. This work offers a scalable, practical pathway to deploying powerful agentic AI systems in real-world settings with favorable cost-to-performance ratios and faster task completion times.

Abstract

Complex agentic AI systems, powered by a coordinated ensemble of Large Language Models (LLMs), tool and memory modules, have demonstrated remarkable capabilities on intricate, multi-turn tasks. However, this success is shadowed by prohibitive economic costs and severe latency, exposing a critical, yet underexplored, trade-off. We formalize this challenge as the \textbf{Agent System Trilemma}: the inherent tension among achieving state-of-the-art performance, minimizing monetary cost, and ensuring rapid task completion. To dismantle this trilemma, we introduce EvoRoute, a self-evolving model routing paradigm that transcends static, pre-defined model assignments. Leveraging an ever-expanding knowledge base of prior experience, EvoRoute dynamically selects Pareto-optimal LLM backbones at each step, balancing accuracy, efficiency, and resource use, while continually refining its own selection policy through environment feedback. Experiments on challenging agentic benchmarks such as GAIA and BrowseComp+ demonstrate that EvoRoute, when integrated into off-the-shelf agentic systems, not only sustains or enhances system performance but also reduces execution cost by up to and latency by over .
Paper Structure (31 sections, 12 equations, 4 figures, 4 tables)

This paper contains 31 sections, 12 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Agentic AI Systems
  4. LLM & Agent Routing
  5. Preliminary
  6. Notations.
  7. Objective Formulation.
  8. Methodology
  9. Self-Evolving Experience Base
  10. Multi-Faceted Retrieval
  11. Pareto-Optimal Filtration and Selection
  12. Experiment
  13. Experiment Setup
  14. Frameworks.
  15. Baselines.
  16. ...and 16 more sections

Figures (4)

  • Figure 1: The agent system trilemma. Existing (deep research) agentic systems excel in certain aspects, yet none of which can fulfill the three characteristics spontaneously.
  • Figure 2: The overview of our proposed EvoRoute.
  • Figure 3: Comparative analysis across three key metrics: performance, cost, and delay, on all subsets of GAIA. All metrics are globally normalized, and values for cost/delay are inverted, such that a larger enclosed area signifies better economy/efficiency.
  • Figure 4: LLM selection distribution of EvoRoute+CK-Pro across different agent roles.