Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Game Intelligence: Theory and Computation

Mehmet Mars Seven

TL;DR

This work introduces the Game Intelligence (GI) mechanism to quantify ex-post strategic intelligence of players in $n$-person games by combining outcomes, engine-based mistakes relative to a reference machine, and player strength. It formalizes a general framework with incomplete knowledge, defines Missed Points (MP), Raw GI, and Expected GI, and introduces concepts such as gamingproofness and dynamic consistency. Empirically, GI is applied to a massive chess dataset (over $10^9$ moves), revealing that Magnus Carlsen achieves the highest GI in world-championship games, with Fischer and Kasparov close in the broader elite set; in engine-vs-engine play, Stockfish leads GI rankings when evaluated against other engines. The paper also develops a standardized GI scale (mean 100, SD 15) and demonstrates robustness across world-championship, elite, regular, and engine-vs-engine data, while discussing theoretical questions about existence of maximally intelligent plays, dynamic consistency, and mechanism consistency. Overall, GI provides a tractable, data-driven framework to assess intelligence in competitive settings and suggests potential applications to AI training and cross-domain game analysis.

Abstract

In this paper, I formalize intelligence measurement in games by introducing mechanisms that assign a real number -- interpreted as an intelligence score -- to each player in a game. This score quantifies the ex-post strategic ability of the players based on empirically observable information, such as the actions of the players, the game's outcome, strength of the players, and a reference oracle machine such as a chess-playing artificial intelligence system. Specifically, I introduce two main concepts: first, the Game Intelligence (GI) mechanism, which quantifies a player's intelligence in a game by considering not only the game's outcome but also the "mistakes" made during the game according to the reference machine's intelligence. Second, I define gamingproofness, a practical and computational concept of strategyproofness. To illustrate the GI mechanism, I apply it to an extensive dataset comprising over a billion chess moves, including over a million moves made by top 20 grandmasters in history. Notably, Magnus Carlsen emerges with the highest GI score among all world championship games included in the dataset. In machine-vs-machine games, the well-known chess engine Stockfish comes out on top.

Game Intelligence: Theory and Computation

TL;DR

This work introduces the Game Intelligence (GI) mechanism to quantify ex-post strategic intelligence of players in -person games by combining outcomes, engine-based mistakes relative to a reference machine, and player strength. It formalizes a general framework with incomplete knowledge, defines Missed Points (MP), Raw GI, and Expected GI, and introduces concepts such as gamingproofness and dynamic consistency. Empirically, GI is applied to a massive chess dataset (over moves), revealing that Magnus Carlsen achieves the highest GI in world-championship games, with Fischer and Kasparov close in the broader elite set; in engine-vs-engine play, Stockfish leads GI rankings when evaluated against other engines. The paper also develops a standardized GI scale (mean 100, SD 15) and demonstrates robustness across world-championship, elite, regular, and engine-vs-engine data, while discussing theoretical questions about existence of maximally intelligent plays, dynamic consistency, and mechanism consistency. Overall, GI provides a tractable, data-driven framework to assess intelligence in competitive settings and suggests potential applications to AI training and cross-domain game analysis.

Abstract

In this paper, I formalize intelligence measurement in games by introducing mechanisms that assign a real number -- interpreted as an intelligence score -- to each player in a game. This score quantifies the ex-post strategic ability of the players based on empirically observable information, such as the actions of the players, the game's outcome, strength of the players, and a reference oracle machine such as a chess-playing artificial intelligence system. Specifically, I introduce two main concepts: first, the Game Intelligence (GI) mechanism, which quantifies a player's intelligence in a game by considering not only the game's outcome but also the "mistakes" made during the game according to the reference machine's intelligence. Second, I define gamingproofness, a practical and computational concept of strategyproofness. To illustrate the GI mechanism, I apply it to an extensive dataset comprising over a billion chess moves, including over a million moves made by top 20 grandmasters in history. Notably, Magnus Carlsen emerges with the highest GI score among all world championship games included in the dataset. In machine-vs-machine games, the well-known chess engine Stockfish comes out on top.
Paper Structure (18 sections, 3 theorems, 34 equations, 1 figure, 10 tables)

This paper contains 18 sections, 3 theorems, 34 equations, 1 figure, 10 tables.

Table of Contents

  1. Introduction
  2. An overview of the computational results
  3. Related work
  4. The theoretical framework
  5. Games
  6. Mechanisms
  7. Response to common theoretical questions
  8. Machine-optimality vs. maximal intelligence
  9. Gamingproofness
  10. Dynamic consistency of machines
  11. Consistency of mechanisms
  12. The dataset, computational methods, and results
  13. World championships and elite tournaments
  14. The top 3 vs other grandmasters
  15. Regular players
  16. ...and 3 more sections

Key Result

Proposition 1

Fix a game $\Gamma$ and a machine $M$. Let $\bar{a}$ be a subplay on the path $[x]$ such that $a_i(x)$, where $i=I(x)$, is human-optimal at node $x$. The EGI mechanism is gamingproof for player $i=I(x)$ at node $x\in X$, if for every $a'_i(x)$ the following inequality holds:

Figures (1)

  • Figure 1: GI score distributions of Fischer, Carlsen, and Kasparov in elite competitions

Theorems & Definitions (17)

  • Definition 1: Machine-optimal actions
  • Definition 2: Missed points
  • Definition 3: Raw Game Intelligence
  • Definition 4: Expected GI
  • Remark 1: Existence of machine-optimal actions
  • proof
  • Definition 5: Maximal intelligence
  • Remark 2: Existence of maximally intelligent plays
  • Definition 6: Gamingproofness
  • Proposition 1
  • ...and 7 more