Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OrchMAS: Orchestrated Reasoning with Multi Collaborative Heterogeneous Scientific Expert Structured Agents

Yichao Feng, Haoran Luo, Zhenghong Lin, Yiqun Sun, Pengfei Wei, Lawrence B. Hsieh, Anh Tuan Luu

TL;DR

This work proposes a scientific domain oriented interactive two tier multi model orchestration framework that supports heterogeneous LLM integration with different capacities or costs, enabling flexible performance efficiency trade offs in practical scientific deployments.

Abstract

Multi-agent large language model frameworks are promising for complex multi step reasoning, yet existing systems remain weak for scientific and knowledge intensive domains due to static prompts and agent roles, rigid workflows, and homogeneous model reliance, leading to poor domain adaptation, limited reasoning flexibility, and high latency on heterogeneous or long-horizon scientific tasks. They also struggle to revise earlier decisions when intermediate reasoning diverges, reducing reliability in structured and calculation heavy settings. To address these limitations, we propose a scientific domain oriented interactive two tier multi model orchestration framework. A dedicated orchestration model analyzes each task, dynamically constructs a domain aware reasoning pipeline, and instantiates specialized expert agents with tailored prompts, while an execution model performs each step under generated role and instruction specifications. The orchestrator iteratively updates the pipeline based on intermediate feedback, enabling dynamic replanning, role reallocation, and prompt refinement across multi turn interactions, strengthening robustness and specialization for scientific reasoning through structured heterogeneous model collaboration. The framework is model agnostic and supports heterogeneous LLM integration with different capacities or costs, enabling flexible performance efficiency trade offs in practical scientific deployments. Experiments show consistent improvements over existing multi agent systems and strong baselines across diverse reasoning and scientific style benchmarks.

OrchMAS: Orchestrated Reasoning with Multi Collaborative Heterogeneous Scientific Expert Structured Agents

TL;DR

This work proposes a scientific domain oriented interactive two tier multi model orchestration framework that supports heterogeneous LLM integration with different capacities or costs, enabling flexible performance efficiency trade offs in practical scientific deployments.

Abstract

Multi-agent large language model frameworks are promising for complex multi step reasoning, yet existing systems remain weak for scientific and knowledge intensive domains due to static prompts and agent roles, rigid workflows, and homogeneous model reliance, leading to poor domain adaptation, limited reasoning flexibility, and high latency on heterogeneous or long-horizon scientific tasks. They also struggle to revise earlier decisions when intermediate reasoning diverges, reducing reliability in structured and calculation heavy settings. To address these limitations, we propose a scientific domain oriented interactive two tier multi model orchestration framework. A dedicated orchestration model analyzes each task, dynamically constructs a domain aware reasoning pipeline, and instantiates specialized expert agents with tailored prompts, while an execution model performs each step under generated role and instruction specifications. The orchestrator iteratively updates the pipeline based on intermediate feedback, enabling dynamic replanning, role reallocation, and prompt refinement across multi turn interactions, strengthening robustness and specialization for scientific reasoning through structured heterogeneous model collaboration. The framework is model agnostic and supports heterogeneous LLM integration with different capacities or costs, enabling flexible performance efficiency trade offs in practical scientific deployments. Experiments show consistent improvements over existing multi agent systems and strong baselines across diverse reasoning and scientific style benchmarks.
Paper Structure (25 sections, 15 equations, 6 figures, 7 tables)

This paper contains 25 sections, 15 equations, 6 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Adaptive MAS Coordination
  5. Layered Critique Refinement Learning
  6. Experiments
  7. Experimental Setup
  8. OrchMAS's performance (RQ1)
  9. Various Task performance (RQ2)
  10. Long Context Performance (RQ3)
  11. Performance On Unseen Tasks (RQ4)
  12. Contribution of Each Elements (RQ5)
  13. Conclusion
  14. Sample Prompt
  15. Coordinator-to-Agent Prompt Template
  16. ...and 10 more sections

Figures (6)

  • Figure 1: Comparison of multi step and MAS LLM generation versus basic prompting.
  • Figure 2: Structural comparison of representative MAS frameworks. Predefined MAS and Selective MAS rely on static role templates and predefined agent libraries, while Auto MAS learns coordination through SFT/RL with a trained agent policy. Interactive Auto MAS performs multi turn dynamic agent and prompt generation and feedback guided orchestration.
  • Figure 3: Training workflow of the OrchMAS framework. The orchestration policy iteratively performs agent role assignment, agent structure construction, and agent prompt generation to dynamically instantiate task specific agents. The process repeats until a finish action is triggered. Layer wise rewards are computed from intermediate trajectories and final answers.
  • Figure 4: Result comparison for easy and hard questions
  • Figure 5: Result comparison for easy and hard questions
  • ...and 1 more figures