Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deploy-Master: Automating the Deployment of 50,000+ Agent-Ready Scientific Tools in One Day

Yi Wang, Zhenting Huang, Zhaohan Ding, Ruoxue Liao, Yuan Huang, Xinzijian Liu, Jiajun Xie, Siheng Chen, Linfeng Zhang

TL;DR

The paper tackles the deployment bottleneck that impedes reproducibility and scalable evaluation of open-source scientific software, a bottleneck that impedes AI-for-Science and agentic workflows. It introduces Deploy-Master, an end-to-end agentic pipeline that first discovers tools at scale, infers executable build specifications, validates execution in containerized runtimes, and publishes runnable capabilities to SciencePedia. In a single day, the approach conducted 52,550 build attempts and produced 50,112 execution-validated tools, accompanied by a large-scale deployment trace that exposes throughput, cost, failure modes, and specification uncertainty. The work demonstrates the feasibility of turning repositories into a shared, execution-grounded substrate for AI4S and agentic science, and outlines a roadmap for evolving toward shared execution environments and governance across tens of thousands of tools.

Abstract

Open-source scientific software is abundant, yet most tools remain difficult to compile, configure, and reuse, sustaining a small-workshop mode of scientific computing. This deployment bottleneck limits reproducibility, large-scale evaluation, and the practical integration of scientific tools into modern AI-for-Science (AI4S) and agentic workflows. We present Deploy-Master, a one-stop agentic workflow for large-scale tool discovery, build specification inference, execution-based validation, and publication. Guided by a taxonomy spanning 90+ scientific and engineering domains, our discovery stage starts from a recall-oriented pool of over 500,000 public repositories and progressively filters it to 52,550 executable tool candidates under license- and quality-aware criteria. Deploy-Master transforms heterogeneous open-source repositories into runnable, containerized capabilities grounded in execution rather than documentation claims. In a single day, we performed 52,550 build attempts and constructed reproducible runtime environments for 50,112 scientific tools. Each successful tool is validated by a minimal executable command and registered in SciencePedia for search and reuse, enabling direct human use and optional agent-based invocation. Beyond delivering runnable tools, we report a deployment trace at the scale of 50,000 tools, characterizing throughput, cost profiles, failure surfaces, and specification uncertainty that become visible only at scale. These results explain why scientific software remains difficult to operationalize and motivate shared, observable execution substrates as a foundation for scalable AI4S and agentic science.

Deploy-Master: Automating the Deployment of 50,000+ Agent-Ready Scientific Tools in One Day

TL;DR

The paper tackles the deployment bottleneck that impedes reproducibility and scalable evaluation of open-source scientific software, a bottleneck that impedes AI-for-Science and agentic workflows. It introduces Deploy-Master, an end-to-end agentic pipeline that first discovers tools at scale, infers executable build specifications, validates execution in containerized runtimes, and publishes runnable capabilities to SciencePedia. In a single day, the approach conducted 52,550 build attempts and produced 50,112 execution-validated tools, accompanied by a large-scale deployment trace that exposes throughput, cost, failure modes, and specification uncertainty. The work demonstrates the feasibility of turning repositories into a shared, execution-grounded substrate for AI4S and agentic science, and outlines a roadmap for evolving toward shared execution environments and governance across tens of thousands of tools.

Abstract

Open-source scientific software is abundant, yet most tools remain difficult to compile, configure, and reuse, sustaining a small-workshop mode of scientific computing. This deployment bottleneck limits reproducibility, large-scale evaluation, and the practical integration of scientific tools into modern AI-for-Science (AI4S) and agentic workflows. We present Deploy-Master, a one-stop agentic workflow for large-scale tool discovery, build specification inference, execution-based validation, and publication. Guided by a taxonomy spanning 90+ scientific and engineering domains, our discovery stage starts from a recall-oriented pool of over 500,000 public repositories and progressively filters it to 52,550 executable tool candidates under license- and quality-aware criteria. Deploy-Master transforms heterogeneous open-source repositories into runnable, containerized capabilities grounded in execution rather than documentation claims. In a single day, we performed 52,550 build attempts and constructed reproducible runtime environments for 50,112 scientific tools. Each successful tool is validated by a minimal executable command and registered in SciencePedia for search and reuse, enabling direct human use and optional agent-based invocation. Beyond delivering runnable tools, we report a deployment trace at the scale of 50,000 tools, characterizing throughput, cost profiles, failure surfaces, and specification uncertainty that become visible only at scale. These results explain why scientific software remains difficult to operationalize and motivate shared, observable execution substrates as a foundation for scalable AI4S and agentic science.
Paper Structure (15 sections, 3 figures)

This paper contains 15 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Deploy-Master System Overview
  3. Tool Discovery at Scale (Search Agent)
  4. Multi-Stage Candidate Funnel.
  5. Automated Build and Validation Pipeline (Build Agent)
  6. Execution-Centered Deployment Under Imperfect Information
  7. Results and Discussion: A Large-Scale Deployment Trace of 50,000+ Scientific Tools
  8. One-Stop Deployment Outcomes and Cost Profile
  9. Corpus Composition, Language Scale, and Deployment Robustness
  10. Failure Surface and Debugging Signals at Scale
  11. Build-System Artifacts as a Scalability Proxy
  12. Outlook: Toward a Shared Execution Substrate for Agentic Science at Scale
  13. From Runnable Tools to Shared Scientific Environments
  14. Implications for Agentic Science and AI4S Evaluation
  15. Limitations and Ongoing Work

Figures (3)

  • Figure 1: Landscape of scientific tools constructed in this work, organized by scientific and engineering application domains. Domain assignment is performed by embedding-based similarity matching between tool descriptions and domain definitions; the visualization uses a similarity threshold of 0.5. Tools may belong to multiple categories due to overlapping functionalities across tasks and disciplines; therefore, the summed counts across categories exceed the total number of unique tools.
  • Figure 2: Deploy-Master workflow from large-scale discovery to runnable scientific capabilities. (A) Search Agent: taxonomy-guided retrieval (91 domains) and iterative expansion produce a recall-oriented pool ($>$500,000), which is progressively filtered to 240,645 tool-like repositories and then to 52,550 executable tool candidates. (B) Build Agent: each candidate is processed via repository analysis, build-specification proposal, dual-model evaluation and refinement, container construction, and execution validation, yielding 50,112 execution-validated tools (95.36% success rate) that are published and registered for search and reuse.
  • Figure 3: Summary of large-scale deployment outcomes and corpus characteristics. The figure aggregates (top left) overall deployment success and failure rates, (top middle) license distribution, (top right) programming language distribution with deployment success rates, (bottom left) failure categories, (bottom middle) outcomes by application level, and (bottom right) deployment success rate versus language scale.