FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks

TL;DR

FrontierScience tackles the need for an unsaturated, expert-level scientific reasoning benchmark by introducing two tracks—Olympiad and Research—that target constrained problem solving and open-ended research tasks across physics, chemistry, and biology. The benchmark combines data collection by domain experts, a four-stage verification pipeline, and a rubric-based scoring framework, with a gold open-source subset of 100 Olympiad and 60 Research questions. Initial results show GPT-5.2 achieving $77%$ on Olympiad and $25%$ on Research, indicating substantial progress on closed-form reasoning but notable gaps in open-ended scientific inquiry. This benchmark provides a diagnostic, cross-domain platform to guide the development of frontier language models toward authentic scientific reasoning and accelerated research capabilities.

Abstract

We introduce FrontierScience, a benchmark evaluating expert-level scientific reasoning in frontier language models. Recent model progress has nearly saturated existing science benchmarks, which often rely on multiple-choice knowledge questions or already published information. FrontierScience addresses this gap through two complementary tracks: (1) Olympiad, consisting of international olympiad problems at the level of IPhO, IChO, and IBO, and (2) Research, consisting of PhD-level, open-ended problems representative of sub-tasks in scientific research. FrontierScience contains several hundred questions (including 160 in the open-sourced gold set) covering subfields across physics, chemistry, and biology, from quantum electrodynamics to synthetic organic chemistry. All Olympiad problems are originally produced by international Olympiad medalists and national team coaches to ensure standards of difficulty, originality, and factuality. All Research problems are research sub-tasks written and verified by PhD scientists (doctoral candidates, postdoctoral researchers, or professors). For Research, we introduce a granular rubric-based evaluation framework to assess model capabilities throughout the process of solving a research task, rather than judging only a standalone final answer.

Figures (11)

• Figure 1: Sample FrontierScience-Olympiad problems. Each task in FrontierScience is written and verified by a domain expert in physics, chemistry, or biology. For the Olympiad set, all experts achieved a medal in an international olympiad competition.
• Figure 2: Sample FrontierScience-Research problems. For the Research set, all experts hold a relevant PhD degree. The corresponding rubrics to these sample tasks can be found in Appendix \ref{['appendix:samples']}.
• Figure 3: Tasks go through four stages: Creation, Review, Resolution, Revision. Independent experts review each other tasks to verify it aligns with the criteria.
• Figure 4: Each task in the research set is graded using a rubric totaling 10 points that can be used by an expert or a judge model. To scale our ability to evaluate models, we use another model to judge responses.
• Figure 5: The Olympiad split is composed of a diverse set of topics, from biochemistry to quantum mechanics.