UNO Arena for Evaluating Sequential Decision-Making Capability of Large Language Models

TL;DR

This work introduces UNO Arena, a dynamic benchmark to evaluate sequential decision-making in large language models by embedding them as UNO players and measuring performance with Monte Carlo–based metrics. It defines three key metrics—$WR$, $ODHR@K$, and $ADR@K$—and introduces TuTri, a reflection-enhanced LLM agent with two reasoning modules that analyze game history and strategy. Empirical results show GPT-4 leading among vanilla LLMs, while the TuTri approach yields notable gains over vanilla LLMs, particularly in 1v1 and 5-player settings, evidencing the value of reflective reasoning in dynamic environments. The paper also analyzes the relationships among the proposed metrics and presents case studies illustrating decision-making dynamics, highlighting implications for evaluating and enhancing LLMs’ planning and strategy in sequential tasks.

Abstract

Sequential decision-making refers to algorithms that take into account the dynamics of the environment, where early decisions affect subsequent decisions. With large language models (LLMs) demonstrating powerful capabilities between tasks, we can't help but ask: Can Current LLMs Effectively Make Sequential Decisions? In order to answer this question, we propose the UNO Arena based on the card game UNO to evaluate the sequential decision-making capability of LLMs and explain in detail why we choose UNO. In UNO Arena, We evaluate the sequential decision-making capability of LLMs dynamically with novel metrics based Monte Carlo methods. We set up random players, DQN-based reinforcement learning players, and LLM players (e.g. GPT-4, Gemini-pro) for comparison testing. Furthermore, in order to improve the sequential decision-making capability of LLMs, we propose the TUTRI player, which can involves having LLMs reflect their own actions wtih the summary of game history and the game strategy. Numerous experiments demonstrate that the TUTRI player achieves a notable breakthrough in the performance of sequential decision-making compared to the vanilla LLM player.

Figures (12)

• Figure 1: In this figure, (A) demonstrates the sequential decision-making process in UNO Arena, (B) shows the execution process of the vanilla LLM player, and (C) shows the execution process of the TuTri player. In fact, The Module (D) and the Module (E) are completely identical.
• Figure 2: The Pearson Correlation Heatmap among WR, ODHR@K (K=2,3,4), and ADR@K (K=2,3,4).
• Figure 3: GPT-4 (the vanilla LLM player) real-time winning rate variations on 4 decks.
• Figure 4: The input1 prompt of the select card shared by the vanilla LLM player and the TuTri player.
• Figure 5: The game history reflection module prompt of the select card for TuTri player.