Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explainable Model Routing for Agentic Workflows

Mika Okamoto, Ansel Kaplan Erol, Mark Riedl

Abstract

Modern agentic workflows decompose complex tasks into specialized subtasks and route them to diverse models to minimize cost without sacrificing quality. However, current routing architectures focus exclusively on performance optimization, leaving underlying trade-offs between model capability and cost unrecorded. Without clear rationale, developers cannot distinguish between intelligent efficiency -- using specialized models for appropriate tasks -- and latent failures caused by budget-driven model selection. We present Topaz, a framework that introduces formal auditability to agentic routing. Topaz replaces silent model assignments with an inherently interpretable router that incorporates three components: (i) skill-based profiling that synthesizes performance across diverse benchmarks into granular capability profiles (ii) fully traceable routing algorithms that utilize budget-based and multi-objective optimization to produce clear traces of how skill-match scores were weighed against costs, and (iii) developer-facing explanations that translate these traces into natural language, allowing users to audit system logic and iteratively tune the cost-quality tradeoff. By making routing decisions interpretable, Topaz enables users to understand, trust, and meaningfully steer routed agentic systems.

Explainable Model Routing for Agentic Workflows

Abstract

Modern agentic workflows decompose complex tasks into specialized subtasks and route them to diverse models to minimize cost without sacrificing quality. However, current routing architectures focus exclusively on performance optimization, leaving underlying trade-offs between model capability and cost unrecorded. Without clear rationale, developers cannot distinguish between intelligent efficiency -- using specialized models for appropriate tasks -- and latent failures caused by budget-driven model selection. We present Topaz, a framework that introduces formal auditability to agentic routing. Topaz replaces silent model assignments with an inherently interpretable router that incorporates three components: (i) skill-based profiling that synthesizes performance across diverse benchmarks into granular capability profiles (ii) fully traceable routing algorithms that utilize budget-based and multi-objective optimization to produce clear traces of how skill-match scores were weighed against costs, and (iii) developer-facing explanations that translate these traces into natural language, allowing users to audit system logic and iteratively tune the cost-quality tradeoff. By making routing decisions interpretable, Topaz enables users to understand, trust, and meaningfully steer routed agentic systems.

Paper Structure

This paper contains 25 sections, 4 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Design and Methods
  4. Skill-Based Profiles for Understanding Models and Agentic Workflows
  5. Cost Models for API-based LLM Inference
  6. Routing Algorithms for Interpretable Task-to-Model Assignment
  7. Skill Match Score.
  8. Explanation Generation
  9. System Demonstration and Case Study
  10. Experimental Setup
  11. Case Study: Customer Support Escalation
  12. Conclusions
  13. Prompts for Topaz
  14. Prompt: Benchmark to Skills
  15. Prompt: Subtask to Skills
  16. ...and 10 more sections

Figures (2)

  • Figure 1: Topaz Architecture. Public benchmarks are synthesized to form model capability profiles. Then, for a new agentic workflow, each subtask is analyzed for complexity and skill requirements. The Topaz routing engine balances skill match and cost, yielding model assignments for each subtask while providing explainable traces for developers.
  • Figure 2: Customer Support Agentic Workflow with colors representing quality sensitivity. The AI Agent parses and classifies tickets, queries relevant documents, and reasons to form a technical diagnosis, lastly escalating or directly responding to the customer.