Multi-Excitation Projective Simulation with a Many-Body Physics Inspired Inductive Bias

TL;DR

This work introduces Multi-Excitation Projective Simulation (MEPS), a quantum-inspired, XAI-friendly extension of Projective Simulation that uses multiple excitations on a hypergraph to model composite thoughts. A physics-inspired inductive bias reduces naive exponential complexity to polynomial in the number of clips, with the degree controlled by the interaction cutoff $IO$, and the training history is captured via a dynamic hypergraph. The authors demonstrate MEPS across three synthetic learning tasks, showing improved interpretability, reduced parameter counts, and faster convergence relative to standard PS and tabular Q-learning baselines. They also outline a path toward quantum MEPS, discussing Hamiltonian-time evolution and potential hardware implementations, and discuss future extensions to real-world data and richer inductive biases for scalable, explainable decision-making. Key contributions include: (i) formalization of MEPS with dynamic hypergraphs, (ii) a provable polynomial complexity reduction via a few-body inductive bias, (iii) empirical demonstrations on three environments illustrating interpretability and efficiency gains, and (iv) a quantum-motivated framework and initial steps toward quantum MEPS.

Abstract

With the impressive progress of deep learning, applications relying on machine learning are increasingly being integrated into daily life. However, most deep learning models have an opaque, oracle-like nature making it difficult to interpret and understand their decisions. This problem led to the development of the field known as eXplainable Artificial Intelligence (XAI). One method in this field known as Projective Simulation (PS) models a chain-of-thought as a random walk of a particle on a graph with vertices that have concepts attached to them. While this description has various benefits, including the possibility of quantization, it cannot be naturally used to model thoughts that combine several concepts simultaneously. To overcome this limitation, we introduce Multi-Excitation Projective Simulation (mePS), a generalization that considers a chain-of-thought to be a random walk of several particles on a hypergraph. A definition for a dynamic hypergraph is put forward to describe the agent's training history along with applications to AI and hypergraph visualization. An inductive bias inspired by the remarkably successful few-body interaction models used in quantum many-body physics is formalized for our classical mePS framework and employed to tackle the exponential complexity associated with naive implementations of hypergraphs. We prove that our inductive bias reduces the complexity from exponential to polynomial, with the exponent representing the cutoff on how many particles can interact. We numerically apply our method to two toy environments and a more complex scenario modelling the diagnosis of a broken computer. These environments demonstrate the resource savings provided by an appropriate choice of inductive bias, as well as showcasing aspects of interpretability. A quantum model for mePS is also briefly outlined and some future directions for it are discussed.

Figures (9)

• Figure 1: An example for the ECM of a PS agent contemplating how to deal with a small ailment. Observations/percepts are shown in blue and actions in red. Furthermore, there are grey internal clips representing intermediate thoughts that lead to a decision.
• Figure 2: An example of a directed, weighted hypergraph describing the ECM of a typical MEPS agent in a reinforcement learning setting. Atomic percept clips are represented in blue with a lowercase $s$, atomic intermediate clips in grey with a lowercase $c$, and atomic action clips in red with a lowercase $a$; the domains and codomains of hyperedges are labelled with capital letters whose clip type corresponds to their lowercase version. Each hyperedge $e \in E$ also has an h-value $h(e)$ associated with it.
• Figure 3: An example illustrating random walk steps under different inductive biases, compare with Example \ref{['Example:InductiveBiases']}. Excited atomic clips are shown in red. The sampled hyperedge is shown in blue. Subfigure a) shows a deliberation step which is only allowed under Inductive Bias \ref{['Bias:ManyBody']}, because its codomain is in two layers. Also, it shows that an excitation moving into an occupied atomic clip gets discarded. Subfigure b) shows a typical transition under Inductive Bias \ref{['Bias:Layered']}.
• Figure 4: The defender $D$ must guess which door the attacker $A$ will go to based on a set of symbols shown to them, and block $A$. $D$ is rewarded for success and punished for failure.
• Figure 5: The average reward learning curves in the Invasion Game with Distraction for MEPS agents with the inductive biases discussed in section \ref{['Subsection:InvasionGameWithDistraction']} and a standard tabular 2-layer Q-learning agent. Note that, in this environment, the 3-body PS is equivalent to standard PS and is used as the comparison of MEPS to standard PS. Each curve is averaged over an ensemble of 50 agents.

Theorems & Definitions (32)

• Definition 1
• Definition 2
• Remark 3
• Definition 4
• Definition 5
• Definition 6
• Definition 7
• Remark 8
• Definition 9
• Definition 10