EEFSUVA: A New Mathematical Olympiad Benchmark
Nicole N Khatibi, Daniil A. Radamovich, Michael P. Brenner
TL;DR
EEFSUVA addresses the question of whether LLMs truly reason through mathematical problems by curating a dataset from underrepresented Eastern European and Soviet Olympiads and Arnold's works, consisting of 45 numerical problems designed to require nonstandard reasoning. The authors compare results from Gemini 2.5 Pro and GPT-5 Thinking against this benchmark, finding a substantial drop in performance relative to Western benchmarks and evidence of brittle, template-based reasoning when problem structure changes. The approach highlights data contamination concerns and the need for broader, cross-tradition evaluation to properly gauge mathematical reasoning in LLMs. The findings suggest that current models still struggle with nonstandard, creativity-demanding problems, motivating future work on broader datasets and training regimes.
Abstract
Recent breakthroughs have spurred claims that large language models (LLMs) match gold medal Olympiad to graduate level proficiency on mathematics benchmarks. In this work, we examine these claims in detail and assess the extent to which current benchmarks capture genuine LLM mathematical reasoning. The composition of these benchmarks, primarily drawing from the International Mathematics Olympiad (IMO) and related competitions, may overstate models reasoning ability due to potential data contamination and a narrow focus on familiar problem types. To enable a more holistic assessment of mathematical understanding, we introduce EEFSUVA, a novel benchmark curated from under circulated regional and national Olympiads of Eastern Europe and the countries from the former Soviet Union. These contests feature problems of comparable difficulty to the IMO and are renowned for demanding nonstandard problem-solving techniques, yet their problems are far less prevalent in online corpora. Preliminary results suggest that even state-of-the-art LLMs exhibit a notable performance decline on EEFSUVA relative to other Olympiad-style benchmarks. These findings also suggest the potential importance of broader evaluation datasets for a fuller assessment of mathematical reasoning and for guiding future model development.