EternalMath: A Living Benchmark of Frontier Mathematics that Evolves with Human Discovery

TL;DR

EternalMath presents a fully automated, theorem-grounded pipeline that converts recent peer-reviewed mathematics into executable, verifiable problem instances, enabling a dynamically evolving benchmark for frontier mathematical reasoning. By filtering papers for recency, authority, and computability and deploying a multi-agent system to generate templates, code, and verification, the approach achieves scalable, low-cost, high-quality evaluation with intrinsic correctness checks. Empirical results show substantial performance gaps for current LLMs on research-level tasks, underscoring the need for benchmarks that evolve with human discovery. The framework promises contamination resistance, domain customization, and sustained relevance as mathematical research progresses. Overall, EternalMath demonstrates a scalable path toward measuring and guiding progress in frontier mathematical reasoning for LLMs.

Abstract

Current evaluations of mathematical reasoning in large language models (LLMs) are dominated by static benchmarks, either derived from competition-style problems or curated through costly expert effort, resulting in limited coverage of research-level mathematics and rapid performance saturation. We propose a fully automated, theorem-grounded pipeline for evaluating frontier mathematical reasoning, which directly transforms recent peer-reviewed mathematical literature into executable and verifiable reasoning tasks. The pipeline identifies constructive or quantitative results, instantiates them into parameterized problem templates, and generates deterministic solutions through execution-based verification, enabling scalable, reproducible, and continuously updatable evaluation without reliance on large-scale expert authoring. By design, this approach supports temporal extensibility, intrinsic correctness checking, and domain-specific customization across mathematical subfields. Applying this pipeline yields \textbf{EternalMath}, an evolving evaluation suite derived from contemporary research papers. Experiments with state-of-the-art LLMs reveal substantial performance gaps, indicating that mathematical reasoning at the research frontier remains far from saturated and underscoring the need for evaluation methodologies that evolve in step with human mathematical discovery.

Figures (6)

• Figure 1: Benchmark discriminative power.(App. \ref{['app:radar']})
• Figure 2: SOTA model performance.
• Figure 3: Overview of the EternalMath construction pipeline. The process consists of four primary stages: (1) Paper Filtering, which selects high-quality, computable research papers from top-tier venues; (2) Multi-agent Collaboration, where specialized agents transform theorems into parameterized meta-templates and executable Python scripts; (3) Execution & Verification, utilizing symbolic computation to ensure deterministic and correct solution generation; and (4) Validation & Quality Control, involving model-based difficulty stratification and human auditing to ensure benchmark rigor and contamination resistance.
• Figure 4: Accuracy of leading large language models on the EternalMath. Models are ranked by their percentage of correctly solved problems, demonstrating a broad spectrum of performance across different model.
• Figure 5: Performance across difficulty tiers. The dramatic decay in accuracy from Easy to Hard levels, underscores the significant challenge EternalMath poses to current models.