Knowledge Retrieval in LLM Gaming: A Shift from Entity-Centric to Goal-Oriented Graphs

TL;DR

The paper tackles the challenge of enabling coherent, multi-step reasoning for LLM-based game agents by replacing fragmented, entity-centric knowledge graphs with Goal-Oriented Graphs (GoGs). GoGs encode goals as nodes with preconditions, postconditions, and hierarchical subgoal dependencies, and employ a two-phase workflow: extract and merge goals from text to build a structured knowledge base, then perform goal-driven retrieval and DFS-based subgoal planning to produce planning prompts for LLMs. Empirical results on a Minecraft testbed show GoG outperforms GraphRAG and baselines, especially on long-horizon tasks, by reducing noise and preserving taskHierarchy for effective planning. The work demonstrates the practical impact of explicit goal hierarchies for reliable, reasoning-enabled agent behavior and suggests broad applicability to other domains requiring structured plan-based reasoning with LLMs.

Abstract

Large Language Models (LLMs) demonstrate impressive general capabilities but often struggle with step-by-step reasoning, especially in complex applications such as games. While retrieval-augmented methods like GraphRAG attempt to bridge this gap through cross-document extraction and indexing, their fragmented entity-relation graphs and overly dense local connectivity hinder the construction of coherent reasoning. In this paper, we propose a novel framework based on Goal-Oriented Graphs (GoGs), where each node represents a goal and its associated attributes, and edges encode logical dependencies between goals. This structure enables explicit retrieval of reasoning paths by first identifying high-level goals and recursively retrieving their subgoals, forming coherent reasoning chains to guide LLM prompting. Our method significantly enhances the reasoning ability of LLMs in game-playing tasks, as demonstrated by extensive experiments on the Minecraft testbed, outperforming GraphRAG and other baselines.

Figures (8)

• Figure 1: GraphRAG extracts an excessive number of low-granularity entity–relation triples, which hinders effective reasoning over fragmented information. In contrast, our GoG captures procedural knowledge through goal hierarchies, thereby supporting coherent and structured reasoning.
• Figure 2: An overview of our proposed method GoG. First, we construct a knowledge base of goals from source text documents. Then, given a task instruction, we retrieve goal-oriented knowledge from the knowledge base to use for plan generation.
• Figure 3: The query pipeline. For a given task, the top-$k$ goals based on embedding similarity between the query and the goals' names are retrieved.
• Figure 4: The LLM used during goal selection and planning stage.
• Figure 5: The prompt used to extract goals and subgoals from source texts to build our knowledge base.