Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

APEX-Agents

Bertie Vidgen, Austin Mann, Abby Fennelly, John Wright Stanly, Lucas Rothman, Marco Burstein, Julien Benchek, David Ostrofsky, Anirudh Ravichandran, Debnil Sur, Neel Venugopal, Alannah Hsia, Isaac Robinson, Calix Huang, Olivia Varones, Daniyal Khan, Michael Haines, Zach Richards, Chirag Mahapatra, Brendan Foody, Osvald Nitski

Abstract

We introduce the AI Productivity Index for Agents (APEX-Agents), a benchmark for assessing whether AI agents can execute long-horizon, cross-application tasks created by investment banking analysts, management consultants, and corporate lawyers. APEX-Agents requires agents to navigate realistic work environments with files and tools. We test eight agents for the leaderboard using Pass@1. Gemini 3 Flash (Thinking=High) achieves the highest score of 24.0%, followed by GPT-5.2 (Thinking=High), Claude Opus 4.5 (Thinking=High), and Gemini 3 Pro (Thinking=High). We open source the APEX-Agents benchmark (n=480) with all prompts, rubrics, gold outputs, files, and metadata. We also open-source Archipelago, our infrastructure for agent execution and evaluation.

APEX-Agents

Abstract

We introduce the AI Productivity Index for Agents (APEX-Agents), a benchmark for assessing whether AI agents can execute long-horizon, cross-application tasks created by investment banking analysts, management consultants, and corporate lawyers. APEX-Agents requires agents to navigate realistic work environments with files and tools. We test eight agents for the leaderboard using Pass@1. Gemini 3 Flash (Thinking=High) achieves the highest score of 24.0%, followed by GPT-5.2 (Thinking=High), Claude Opus 4.5 (Thinking=High), and Gemini 3 Pro (Thinking=High). We open source the APEX-Agents benchmark (n=480) with all prompts, rubrics, gold outputs, files, and metadata. We also open-source Archipelago, our infrastructure for agent execution and evaluation.
Paper Structure (36 sections, 4 figures, 10 tables)

This paper contains 36 sections, 4 figures, 10 tables.

Table of Contents

  1. Introduction
  2. APEX Survey
  3. Work activities
  4. Benchmark design and dataset
  5. World creation
  6. Tasks
  7. Rubrics and gold outputs
  8. Contributors to APEX– Agents
  9. Baselining study
  10. Evaluation
  11. Collecting agent outputs
  12. Judge model
  13. Metrics
  14. Results on APEX– Agents
  15. Pass@1 (Leaderboard)
  16. ...and 21 more sections

Figures (4)

  • Figure 1: Performance of agents on the APEX– Agents benchmark using Pass@1. Thinking settings are in parentheses.
  • Figure 2: Performance of agents on the APEX– Agents benchmark using Pass@1, segmented by job. Thinking settings are in parentheses.
  • Figure 3: Overview of the approach in APEX– Agents to agentic AI evaluation.
  • Figure 4: Pass@$k$ and Pass^k of agents on the APEX– Agents benchmark. Solid lines indicate Pass@$k$, and dotted lines indicate Pass^k.