Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EconWebArena: Benchmarking Autonomous Agents on Economic Tasks in Realistic Web Environments

Zefang Liu, Yinzhu Quan

TL;DR

EconWebArena introduces a realistic benchmark for autonomous web agents to retrieve precise economic data from live, authoritative websites. It assembles 360 tasks across 82 sources, generated via multiple LLMs and curated for factual grounding, with evaluation requiring exact numeric answers and correct source domains. The framework combines BrowserGym-based observation and action spaces with multimodal reasoning prompts, enabling rigorous testing of grounding, navigation, and data extraction capabilities. Experimental results show substantial gaps to human performance and identify planned directions—visual grounding, planning, and hybrid GUI/API strategies—as promising paths to improve domain-aware web intelligence in economic contexts.

Abstract

We introduce EconWebArena, a benchmark for evaluating autonomous agents on complex, multimodal economic tasks in realistic web environments. The benchmark comprises 360 curated tasks from 82 authoritative websites spanning domains such as macroeconomics, labor, finance, trade, and public policy. Each task challenges agents to navigate live websites, interpret structured and visual content, interact with real interfaces, and extract precise, time-sensitive data through multi-step workflows. We construct the benchmark by prompting multiple large language models (LLMs) to generate candidate tasks, followed by rigorous human curation to ensure clarity, feasibility, and source reliability. Unlike prior work, EconWebArena emphasizes fidelity to authoritative data sources and the need for grounded web-based economic reasoning. We evaluate a diverse set of state-of-the-art multimodal LLMs as web agents, analyze failure cases, and conduct ablation studies to assess the impact of visual grounding, plan-based reasoning, and interaction design. Our results reveal substantial performance gaps and highlight persistent challenges in grounding, navigation, and multimodal understanding, positioning EconWebArena as a rigorous testbed for economic web intelligence.

EconWebArena: Benchmarking Autonomous Agents on Economic Tasks in Realistic Web Environments

TL;DR

EconWebArena introduces a realistic benchmark for autonomous web agents to retrieve precise economic data from live, authoritative websites. It assembles 360 tasks across 82 sources, generated via multiple LLMs and curated for factual grounding, with evaluation requiring exact numeric answers and correct source domains. The framework combines BrowserGym-based observation and action spaces with multimodal reasoning prompts, enabling rigorous testing of grounding, navigation, and data extraction capabilities. Experimental results show substantial gaps to human performance and identify planned directions—visual grounding, planning, and hybrid GUI/API strategies—as promising paths to improve domain-aware web intelligence in economic contexts.

Abstract

We introduce EconWebArena, a benchmark for evaluating autonomous agents on complex, multimodal economic tasks in realistic web environments. The benchmark comprises 360 curated tasks from 82 authoritative websites spanning domains such as macroeconomics, labor, finance, trade, and public policy. Each task challenges agents to navigate live websites, interpret structured and visual content, interact with real interfaces, and extract precise, time-sensitive data through multi-step workflows. We construct the benchmark by prompting multiple large language models (LLMs) to generate candidate tasks, followed by rigorous human curation to ensure clarity, feasibility, and source reliability. Unlike prior work, EconWebArena emphasizes fidelity to authoritative data sources and the need for grounded web-based economic reasoning. We evaluate a diverse set of state-of-the-art multimodal LLMs as web agents, analyze failure cases, and conduct ablation studies to assess the impact of visual grounding, plan-based reasoning, and interaction design. Our results reveal substantial performance gaps and highlight persistent challenges in grounding, navigation, and multimodal understanding, positioning EconWebArena as a rigorous testbed for economic web intelligence.

Paper Structure

This paper contains 26 sections, 14 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Web-Based Benchmarks
  4. Economic Benchmarks
  5. EconWebArena
  6. Task Generation
  7. Task Curation and Annotation
  8. Observation and Action Spaces
  9. Answer Evaluation
  10. Experiments
  11. Experimental Setup
  12. Experimental Results
  13. RQ1: How well can LLM agents solve economic tasks in realistic web environments?
  14. RQ2: What causes LLM agents to fail on EconWebArena tasks?
  15. RQ3: What directions can improve LLM agent performance on these tasks?
  16. ...and 11 more sections

Figures (14)

  • Figure 1: Overview of EconWebArena where agents solve realistic economic tasks by navigating real websites, interpreting content, and extracting grounded numeric answers.
  • Figure 2: Illustrative example of a typical EconWebArena task. Solving a task involves multi-step navigation across economic data websites, accessing the appropriate report or database, and extracting a target numeric value from tables, diagrams, or databases.
  • Figure 3: Representative EconWebArena tasks involve interacting with diverse web elements, including maps, diagrams, tables, reports, databases, documents, and so on.
  • Figure 4: Distribution of task categories in EconWebArena, reflecting the diversity of economic domains represented in the benchmark.
  • Figure 5: Representative examples of common error types from o4-mini in EconWebArena: access issues, data extraction errors, navigation failures, visual understanding failures, and interaction failures.
  • ...and 9 more figures