Toward a Metrology for Artificial Intelligence: Hidden-Rule Environments and Reinforcement Learning

TL;DR

This work probes AI metrology by using the GOHR environment to study how agents infer hidden rules under partial observations. It compares Feature-Centric and Object-Centric state encodings fed into a Transformer-based A2C policy/critic, assessing learning efficiency, transfer, and generalization across diverse rule sets. Through independent-rule, transfer, and generalization experiments, the study reveals that OC representations yield more stable cross-metric difficulty and better generalization, while FC representations exhibit metric-dependent variability; transfer effects indicate that learning component rules speeds up compound-rule acquisition when pretraining is comprehensive. The results motivate a formal notion of a difficulty space for hidden-rule tasks and suggest a path toward a Cognodynamics-like science for measuring and teaching artificial intelligence.

Abstract

We investigate reinforcement learning in the Game Of Hidden Rules (GOHR) environment, a complex puzzle in which an agent must infer and execute hidden rules to clear a 6$\times$6 board by placing game pieces into buckets. We explore two state representation strategies, namely Feature-Centric (FC) and Object-Centric (OC), and employ a Transformer-based Advantage Actor-Critic (A2C) algorithm for training. The agent has access only to partial observations and must simultaneously infer the governing rule and learn the optimal policy through experience. We evaluate our models across multiple rule-based and trial-list-based experimental setups, analyzing transfer effects and the impact of representation on learning efficiency.

Figures (14)

• Figure 1: Transformer architecture used for policy network and critic network.
• Figure 2: Feature-Centric (FC) representation of a single board at a given time step. The board is encoded as a stacked one-hot tensor with 8 feature maps, each of size $6 \times 6$. Features $f_1$–$f_8$ correspond to square, star, circle, triangle, red, black, blue, and yellow, respectively. The example shows the placement of 9 objects, such as a red triangle at position 1 and a yellow star at position 9.
• Figure 3: Object-Centric (OC) representation of a single board at a given time step. The board is encoded as a one-hot tensor with 9 rows, each corresponding to one object on the board. Every object is represented as a 24-dimensional one-hot vector, partitioned into color, shape, $x$, and $y$ components. The grouping illustrates how these components form the OC model input representation.
• Figure 4: Feature construction pipeline of FC model. Top: Past state formed by concatenating the flattened FC state ($\mathbb{R}^{288}$) with the action encoding ($\mathbb{R}^{144}$), yielding $\mathbb{R}^{432}$. Bottom: Current input$_{\text{FC}}$ formed by concatenating the current flattened board ($\mathbb{R}^{288}$) with six past states ($6\times \mathbb{R}^{432}$), producing $\mathbb{R}^{2880}$.
• Figure 5: Feature construction pipeline of OC model. $m$ is the number of objects, n is the number of previous board states.