BABE: Biology Arena BEnchmark
Junting Zhou, Jin Chen, Linfeng Hao, Denghui Cao, Zheyu Wang, Qiguang Chen, Chaoyou Fu, Jiaze Chen, Yuchen Wu, Ge Zhang, Mingxuan Wang, Wenhao Huang, Tong Yang
TL;DR
BABE addresses a gap in biology AI benchmarks by focusing on experimental reasoning that integrates results with context. It frames evaluation around a single source document $D$ using a triplet of questions $Q_{BABE}$ with correlation relations to diagnose sequential versus parallel reasoning. The data pipeline blends frontier literature curation, expert item development, and multi-stage quality assurance to produce high-quality, research-derived tasks across 12 biology subfields. Findings show that deeper, sustained reasoning and multi-trial inference improve performance, highlighting the benchmark's potential to guide the development of biologically grounded, discovery-oriented AI.
Abstract
The rapid evolution of large language models (LLMs) has expanded their capabilities from basic dialogue to advanced scientific reasoning. However, existing benchmarks in biology often fail to assess a critical skill required of researchers: the ability to integrate experimental results with contextual knowledge to derive meaningful conclusions. To address this gap, we introduce BABE(Biology Arena BEnchmark), a comprehensive benchmark designed to evaluate the experimental reasoning capabilities of biological AI systems. BABE is uniquely constructed from peer-reviewed research papers and real-world biological studies, ensuring that tasks reflect the complexity and interdisciplinary nature of actual scientific inquiry. BABE challenges models to perform causal reasoning and cross-scale inference. Our benchmark provides a robust framework for assessing how well AI systems can reason like practicing scientists, offering a more authentic measure of their potential to contribute to biological research.