Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMs for Game Theory: Entropy-Guided In-Context Learning and Adaptive CoT Reasoning

Tommaso Felice Banfi, Sashenka Gamage

TL;DR

This work tackles sequential decision-making with LLMs by introducing entropy-guided context retrieval and adaptive chain-of-thought reasoning. The approach retrieves context based on token-level uncertainty and adaptively expands reasoning paths, enabling efficient, uncertainty-aware planning in Tic-Tac-Toe. Empirical results show the method elevates average outcomes from $-11.6\%$ to $+9.5\%$ (with fewer queries than strong baselines), and a negative correlation between entropy and move quality confirms uncertainty as a practical signal. The findings suggest uncertainty-aware retrieval and CoT can improve LLMs' performance in structured, multi-step tasks without task-specific fine-tuning, with potential applicability to larger, less predictable environments.

Abstract

We propose a novel LLM-based framework for reasoning in discrete, game-theoretic tasks, illustrated with \emph{Tic-Tac-Toe}. The method integrates in-context learning with entropy-guided chain-of-thought (CoT) reasoning and adaptive context retrieval. The model dynamically adjusts both the number of retrieved examples and reasoning paths according to token-level uncertainty: concise reasoning with minimal context is used when uncertainty is low, whereas higher uncertainty triggers expanded multi-path CoT exploration. Experimental evaluation against a sub-optimal algorithmic opponent shows that entropy-aware adaptive reasoning substantially improves decision quality, increasing the average game outcome from \(-11.6\%\) with the baseline LLM to \(+9.5\%\) with entropy-guided adaptive reasoning over 100 games (win = +1, tie = 0, loss = -1), while maintaining a relatively low number of LLM queries per game. Statistical validation confirms that the improvement is significant, and correlation analysis reveals a negative association between token-level entropy and move optimality. These findings demonstrate that uncertainty-guided adaptive reasoning effectively enhances LLM performance in sequential decision-making environments.

LLMs for Game Theory: Entropy-Guided In-Context Learning and Adaptive CoT Reasoning

TL;DR

This work tackles sequential decision-making with LLMs by introducing entropy-guided context retrieval and adaptive chain-of-thought reasoning. The approach retrieves context based on token-level uncertainty and adaptively expands reasoning paths, enabling efficient, uncertainty-aware planning in Tic-Tac-Toe. Empirical results show the method elevates average outcomes from to (with fewer queries than strong baselines), and a negative correlation between entropy and move quality confirms uncertainty as a practical signal. The findings suggest uncertainty-aware retrieval and CoT can improve LLMs' performance in structured, multi-step tasks without task-specific fine-tuning, with potential applicability to larger, less predictable environments.

Abstract

We propose a novel LLM-based framework for reasoning in discrete, game-theoretic tasks, illustrated with \emph{Tic-Tac-Toe}. The method integrates in-context learning with entropy-guided chain-of-thought (CoT) reasoning and adaptive context retrieval. The model dynamically adjusts both the number of retrieved examples and reasoning paths according to token-level uncertainty: concise reasoning with minimal context is used when uncertainty is low, whereas higher uncertainty triggers expanded multi-path CoT exploration. Experimental evaluation against a sub-optimal algorithmic opponent shows that entropy-aware adaptive reasoning substantially improves decision quality, increasing the average game outcome from with the baseline LLM to with entropy-guided adaptive reasoning over 100 games (win = +1, tie = 0, loss = -1), while maintaining a relatively low number of LLM queries per game. Statistical validation confirms that the improvement is significant, and correlation analysis reveals a negative association between token-level entropy and move optimality. These findings demonstrate that uncertainty-guided adaptive reasoning effectively enhances LLM performance in sequential decision-making environments.
Paper Structure (23 sections, 18 equations, 1 figure, 1 table)

This paper contains 23 sections, 18 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Definition
  4. Method
  5. Game Loop
  6. Board Representation and Context Retrieval
  7. Vector Database.
  8. Context Construction.
  9. Adaptive Chain-of-Thought Reasoning
  10. Reasoning Modes.
  11. Token-level Entropy and Adaptive Branching.
  12. Experimental Setup
  13. Hyperparameters and Implementation Details.
  14. Results and Discussion
  15. Entropy as a Proxy for Uncertainty.
  16. ...and 8 more sections

Figures (1)

  • Figure 1: Token-level entropy versus move optimality percentile. Top: boxplots grouped into three clusters (Suboptimal, Moderate, Near-Optimal), showing higher entropy for less optimal moves. Bottom: scatter plot of individual moves with a linear regression line, illustrating the negative relationship between entropy and move optimality. Data are based on 500 randomly selected board states, considering all valid moves per state.