Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Can Agent Conquer Web? Exploring the Frontiers of ChatGPT Atlas Agent in Web Games

Jingran Zhang, Ning Li, Justin Cui

TL;DR

The paper evaluates OpenAI's ChatGPT Atlas on web-based games to understand its performance in dynamic interactive tasks. Using a zero-shot, browser-based protocol across five games, it reports strong Sudoku-style logical reasoning but poor real-time motor control in reflex-like tasks, and limited autonomous behavior in RPG navigation. The findings highlight Atlas's analytical strengths alongside notable execution and contextual understanding gaps, suggesting that browser-control capabilities suffice for information tasks but require targeted improvements for real-time interaction and long-horizon planning. This work provides a pragmatic benchmark and actionable directions for advancing generalist web agents toward robust, end-to-end interaction in complex online environments.

Abstract

OpenAI's ChatGPT Atlas introduces new capabilities for web interaction, enabling the model to analyze webpages, process user intents, and execute cursor and keyboard inputs directly within the browser. While its capacity for information retrieval tasks has been demonstrated, its performance in dynamic, interactive environments remains less explored. In this study, we conduct an early evaluation of Atlas's web interaction capabilities using browser-based games as test scenarios, including Google's T-Rex Runner, Sudoku, Flappy Bird, and Stein.world. We employ in-game performance scores as quantitative metrics to assess performance across different task types. Our results show that Atlas performs strongly in logical reasoning tasks like Sudoku, completing puzzles significantly faster than human baselines, but struggles substantially in real-time games requiring precise timing and motor control, often failing to progress beyond initial obstacles. These findings suggest that while Atlas demonstrates capable analytical processing, there remain notable limitations in dynamic web environments requiring real-time interaction. The website of our project can be found at https://atlas-game-eval.github.io.

Can Agent Conquer Web? Exploring the Frontiers of ChatGPT Atlas Agent in Web Games

TL;DR

The paper evaluates OpenAI's ChatGPT Atlas on web-based games to understand its performance in dynamic interactive tasks. Using a zero-shot, browser-based protocol across five games, it reports strong Sudoku-style logical reasoning but poor real-time motor control in reflex-like tasks, and limited autonomous behavior in RPG navigation. The findings highlight Atlas's analytical strengths alongside notable execution and contextual understanding gaps, suggesting that browser-control capabilities suffice for information tasks but require targeted improvements for real-time interaction and long-horizon planning. This work provides a pragmatic benchmark and actionable directions for advancing generalist web agents toward robust, end-to-end interaction in complex online environments.

Abstract

OpenAI's ChatGPT Atlas introduces new capabilities for web interaction, enabling the model to analyze webpages, process user intents, and execute cursor and keyboard inputs directly within the browser. While its capacity for information retrieval tasks has been demonstrated, its performance in dynamic, interactive environments remains less explored. In this study, we conduct an early evaluation of Atlas's web interaction capabilities using browser-based games as test scenarios, including Google's T-Rex Runner, Sudoku, Flappy Bird, and Stein.world. We employ in-game performance scores as quantitative metrics to assess performance across different task types. Our results show that Atlas performs strongly in logical reasoning tasks like Sudoku, completing puzzles significantly faster than human baselines, but struggles substantially in real-time games requiring precise timing and motor control, often failing to progress beyond initial obstacles. These findings suggest that while Atlas demonstrates capable analytical processing, there remain notable limitations in dynamic web environments requiring real-time interaction. The website of our project can be found at https://atlas-game-eval.github.io.

Paper Structure

This paper contains 16 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Experiment
  4. Experimental Setup
  5. Experimental Design
  6. Experimental Observations
  7. T-Rex Runner: Systematic Timing Failures
  8. Flappy Bird: Uncoordinated Motor Control
  9. Sudoku: Efficient Logical Reasoning
  10. 2048: Control Exploration with Minimal Planning
  11. Stein.world: Textual Understanding and Task Execution in RPG games.
  12. Summary of Behavioral Patterns
  13. Operational Challenges
  14. Limitations and Discussion
  15. Conclusion
  16. ...and 1 more sections

Figures (2)

  • Figure 1: ChatGPT Atlas being evaluated across different web games. The agent performs strategic moves in 2048 (Top-Left), logical placements in Sudoku (Top-Right), key presses for timing in T-Rex Runner (Bottom-Left), rapid taps for continuous control in Flappy Bird (Center), and exploration, navigation, game-understanding in Stein.world (Bottom-Right).
  • Figure 2: Layout of the Stein.world starting area and the location of the first task. The player begins in the spawn room and must navigate outside to speak with the NPC Cleaning Lady to advance the storyline. Atlas initially failed to exit the room despite extended attempts, but after receiving detailed operational instructions, it successfully completed the task.