SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?

TL;DR

The paper tackles the challenge of enabling open-source AI agents to drive scientific discovery by introducing X-Master, a tool-augmented reasoning agent that uses code as an interaction language to interface with external tools. It then scales this approach with X-Masters, a scattered-and-stacked inference-time workflow that orchestrates multiple agent roles to broaden exploration and deepen refinement. On Humanity's Last Exam, X-Masters sets a new state-of-the-art of 32.1%, exceeding closed-source benchmarks, and shows strong performance in biology-related tasks and TRQA benchmarks, underscoring the practical potential of open, inference-time tool use. The work emphasizes sharing practical know-how and outlines a roadmap toward end-to-end trained, specialized scientific agents, highlighting the value of open-source architectures for rapid progress in scientific AI.

Abstract

The rapid advancements of AI agents have ignited the long-held ambition of leveraging them to accelerate scientific discovery. Achieving this goal requires a deep understanding of the frontiers of human knowledge. As such, Humanity's Last Exam (HLE) provides an exceptionally challenging touchstone for evaluating scientific AI agents. In this work, we aim to construct the foundational architecture for general-purpose agents and validate the capabilities through leading performance on HLE. To achieve this, we introduce X-Master, a tool-augmented reasoning agent designed to emulate human researchers by interacting flexibly with external tools during its reasoning process. This agent, guided by the conceptualization of code as an interaction language, can flexibly leverage built-in Python libraries and our customized tools to augment the reasoning. We further scale its capabilities through X-Masters, a scattered-and-stacked agentic workflow that systematically enhances breadth and depth of reasoning. Our open-source solution, X-Masters, sets a new state-of-the-art record on HLE with a score of 32.1%, surpassing OpenAI's and Google's Deep Research (26.6% and 26.9%) and becoming the first to exceed the 30% threshold. This work allows us to gain a deeper understanding of complex task-solving and accumulates valuable experience that can inform future advancements, guiding subsequent model training.

Figures (10)

• Figure 1: Comparisons on Humanity's Last Exam. Our X-Masters achieves the state-of-the-art score of 32.1%, surpassing deep research products from Kimi, Gemini, and OpenAI.
• Figure 2: Overview of our X-Master, a tool-augmented reasoning agent. Given a user query, the agent starts the thinking process, where interaction with the environments such as tool calling would be invoked by generating a code snippet. The execution results will be appended to the agent's context, enriching its understanding and informing its subsequent thinking. In this case, the agent invokes interactions for three times (search to get the GitHub link, parse to get the arXiv paper link, then parse to get the affiliation) until it arrives at the final answer.
• Figure 3: Overview of our X-Masters, a scattered-and-stacked agentic workflow. The workflow leverages X-Master as different roles to enhance solution quality at inference. It includes (1) Solvers generating five initial solutions, (2) Critics refining the initial solution, (3) Rewriters synthesizing all five solutions to generate new five, and (4) Selector choosing the best solution.
• Figure 4: Performance across categories of DeepSeek-R1-0528 and X-Masters on HLE.
• Figure 5: Performance of Biology/Medicine category of HLE.