Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Playing NetHack with LLMs: Potential & Limitations as Zero-Shot Agents

Dominik Jeurissen, Diego Perez-Liebana, Jeremy Gow, Duygu Cakmak, James Kwan

TL;DR

NetPlay is presented, the first LLM powered zero-shot agent for the challenging roguelike NetHack and demonstrates the necessity for dynamic methods in supplying context information for complex games such as NetHack.

Abstract

Large Language Models (LLMs) have shown great success as high-level planners for zero-shot game-playing agents. However, these agents are primarily evaluated on Minecraft, where long-term planning is relatively straightforward. In contrast, agents tested in dynamic robot environments face limitations due to simplistic environments with only a few objects and interactions. To fill this gap in the literature, we present NetPlay, the first LLM-powered zero-shot agent for the challenging roguelike NetHack. NetHack is a particularly challenging environment due to its diverse set of items and monsters, complex interactions, and many ways to die. NetPlay uses an architecture designed for dynamic robot environments, modified for NetHack. Like previous approaches, it prompts the LLM to choose from predefined skills and tracks past interactions to enhance decision-making. Given NetHack's unpredictable nature, NetPlay detects important game events to interrupt running skills, enabling it to react to unforeseen circumstances. While NetPlay demonstrates considerable flexibility and proficiency in interacting with NetHack's mechanics, it struggles with ambiguous task descriptions and a lack of explicit feedback. Our findings demonstrate that NetPlay performs best with detailed context information, indicating the necessity for dynamic methods in supplying context information for complex games such as NetHack.

Playing NetHack with LLMs: Potential & Limitations as Zero-Shot Agents

TL;DR

NetPlay is presented, the first LLM powered zero-shot agent for the challenging roguelike NetHack and demonstrates the necessity for dynamic methods in supplying context information for complex games such as NetHack.

Abstract

Large Language Models (LLMs) have shown great success as high-level planners for zero-shot game-playing agents. However, these agents are primarily evaluated on Minecraft, where long-term planning is relatively straightforward. In contrast, agents tested in dynamic robot environments face limitations due to simplistic environments with only a few objects and interactions. To fill this gap in the literature, we present NetPlay, the first LLM-powered zero-shot agent for the challenging roguelike NetHack. NetHack is a particularly challenging environment due to its diverse set of items and monsters, complex interactions, and many ways to die. NetPlay uses an architecture designed for dynamic robot environments, modified for NetHack. Like previous approaches, it prompts the LLM to choose from predefined skills and tracks past interactions to enhance decision-making. Given NetHack's unpredictable nature, NetPlay detects important game events to interrupt running skills, enabling it to react to unforeseen circumstances. While NetPlay demonstrates considerable flexibility and proficiency in interacting with NetHack's mechanics, it struggles with ambiguous task descriptions and a lack of explicit feedback. Our findings demonstrate that NetPlay performs best with detailed context information, indicating the necessity for dynamic methods in supplying context information for complex games such as NetHack.
Paper Structure (18 sections, 2 figures, 3 tables)

This paper contains 18 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. NetHack
  4. NetHack Learning Environment
  5. autoascend
  6. LLM Agents
  7. NetPlay
  8. Prompting
  9. Skills
  10. Agent Loop
  11. Handcrafted Agent
  12. Experiments
  13. Setup
  14. Full Runs
  15. Scenarios
  16. ...and 3 more sections

Figures (2)

  • Figure 1: The terminal view of the game NetHack. The left image presents an annotated view of the in-game screen, featuring the game's map, an example of a game message, and the agent's stats. The right image showcases a menu for picking up items from a tile containing multiple objects. Image source: https://alt.org/nethack/
  • Figure 2: Illustration of NetPlay playing NetHack. The process involves constructing a prompt using messages representing past events, the current observation, and a task description containing available skills and the desired output format. The response is parsed to retrieve the next skill. While executing the selected skill, a tracker enriches the given observations and detects important events, such as when a new monster appears. When the skill is done, or events interrupt the skill execution, the agent will restart the prompting process.