Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models

Mehrzad Samadi, Aleksander Ficek, Sean Narenthiran, Siddhartha Jain, Wasi Uddin Ahmad, Somshubra Majumdar, Vahid Noroozi, Boris Ginsburg

TL;DR

This work tackles the challenge of achieving IOI gold-level performance with open-weight LLMs for competitive programming. It proposes GenCluster, a four-stage test-time compute pipeline—parallel generation, behavioral clustering, tournament-based ranking, and round-robin submission—that efficiently selects top solutions under IOI submission limits. The authors demonstrate that, with GenCluster and gpt-oss-120b, gold-level performance is attainable at IOI 2025, and they provide extensive ablations showing scalable gains with increased compute. The methodology offers a transparent, reproducible baseline for evaluating reasoning in LLMs and charts a practical path to narrow the gap between open-weight and proprietary systems in competitive programming tasks.

Abstract

Competitive programming has become a rigorous benchmark for evaluating the reasoning and problem-solving capabilities of large language models (LLMs). The International Olympiad in Informatics (IOI) stands out as one of the most prestigious annual competitions in competitive programming and has become a key benchmark for comparing human and AI-level programming ability. While several proprietary models have been claimed to achieve gold medal-level performance at the IOI, often with undisclosed methods, achieving comparable results with open-weight models remains a significant challenge. In this paper, we present \gencluster, a scalable and reproducible test-time compute framework that attains IOI gold-level performance using open-weight models. It combines large-scale generation, behavioral clustering, ranking, and a round-robin submission strategy to efficiently explore diverse solution spaces under limited validation budgets. Our experiments show that the performance of our proposed approach scales consistently with available compute, narrowing the gap between open and closed systems. Notably, we will show that GenCluster can achieve a gold medal at IOI 2025 for the first time with an open-weight model gpt-oss-120b, setting a new benchmark for transparent and reproducible evaluation of reasoning in LLMs.

Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models

TL;DR

This work tackles the challenge of achieving IOI gold-level performance with open-weight LLMs for competitive programming. It proposes GenCluster, a four-stage test-time compute pipeline—parallel generation, behavioral clustering, tournament-based ranking, and round-robin submission—that efficiently selects top solutions under IOI submission limits. The authors demonstrate that, with GenCluster and gpt-oss-120b, gold-level performance is attainable at IOI 2025, and they provide extensive ablations showing scalable gains with increased compute. The methodology offers a transparent, reproducible baseline for evaluating reasoning in LLMs and charts a practical path to narrow the gap between open-weight and proprietary systems in competitive programming tasks.

Abstract

Competitive programming has become a rigorous benchmark for evaluating the reasoning and problem-solving capabilities of large language models (LLMs). The International Olympiad in Informatics (IOI) stands out as one of the most prestigious annual competitions in competitive programming and has become a key benchmark for comparing human and AI-level programming ability. While several proprietary models have been claimed to achieve gold medal-level performance at the IOI, often with undisclosed methods, achieving comparable results with open-weight models remains a significant challenge. In this paper, we present \gencluster, a scalable and reproducible test-time compute framework that attains IOI gold-level performance using open-weight models. It combines large-scale generation, behavioral clustering, ranking, and a round-robin submission strategy to efficiently explore diverse solution spaces under limited validation budgets. Our experiments show that the performance of our proposed approach scales consistently with available compute, narrowing the gap between open and closed systems. Notably, we will show that GenCluster can achieve a gold medal at IOI 2025 for the first time with an open-weight model gpt-oss-120b, setting a new benchmark for transparent and reproducible evaluation of reasoning in LLMs.
Paper Structure (20 sections, 11 figures, 2 tables)

This paper contains 20 sections, 11 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Related Work
  4. IOI Competition and Benchmark
  5. Proposed Approach
  6. Parallel Candidates Generation
  7. Behavioral Clustering
  8. Ranking with Tournament
  9. Submission Strategy
  10. Experiments
  11. Performance Comparison between Models
  12. Evaluating GenCluster under Submission Constraints
  13. Comparison with other Test-time Compute Strategies
  14. Impact of the Test Set Size on Clustering
  15. Impact of the Number of Games in Tournament
  16. ...and 5 more sections

Figures (11)

  • Figure 1: The overall pipeline of GenCluster for a single subtask, a process to be repeated for every subtask in the IOI benchmark.
  • Figure 2: Final scores of different models on IOI 2025 when generating $K$ solutions per subtask, assuming unlimited submissions. Medal thresholds are indicated on the chart, and the maximum achievable score is 600.
  • Figure 3: Performance of gpt-oss-120b on IOI 2025 with and without the 50-submission limit, varying generation counts. Results are averaged over five runs (except $K{=}5000$, single run). Medal thresholds are indicated on the chart, and the score reported by OpenAI is shown for comparison.
  • Figure 4: Shown are the cluster purity (F1-score), average cluster size, and average number of clusters for different numbers of test cases. Results are reported for $K = 5000$ using gpt-oss-120b.
  • Figure 5: Effect of the number of games per cluster on final score.
  • ...and 6 more figures