Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Structure and Reduction of MCTS for Explainable-AI

Ronit Bustin, Claudia V. Goldman

TL;DR

This paper tackles explainability for AlphaZero-style MCTS planners by focusing on the information content of the MCTS structure. It introduces an information-theoretic entropy measure over MCTS subtrees and shows how to compute and update this entropy efficiently during the MCTS construction to support explanations. It then proposes entropy-guided subtree removals to balance explainability and tree size, with theoretical results on entropy change under removals and greedy algorithms evaluated in autonomous-driving simulations. The work demonstrates a practical path to generate human-understandable explanations from MCTS data and to scale explainability through controlled tree reduction.

Abstract

Complex sequential decision-making planning problems, covering infinite states' space have been shown to be solvable by AlphaZero type of algorithms. Such an approach that trains a neural model while simulating projection of futures with a Monte Carlo Tree Search algorithm were shown to be applicable to real life planning problems. As such, engineers and users interacting with the resulting policy of behavior might benefit from obtaining automated explanations about these planners' decisions offline or online. This paper focuses on the information within the Monte Carlo Tree Search data structure. Given its construction, this information contains much of the reasoning of the sequential decision-making algorithm and is essential for its explainability. We show novel methods using information theoretic tools for the simplification and reduction of the Monte Carlo Tree Search and the extraction of information. Such information can be directly used for the construction of human understandable explanations. We show that basic explainability quantities can be calculated with limited additional computational cost, as an integrated part of the Monte Carlo Tree Search construction process. We focus on the theoretical and algorithmic aspects and provide examples of how the methods presented here can be used in the construction of human understandable explanations.

Structure and Reduction of MCTS for Explainable-AI

TL;DR

This paper tackles explainability for AlphaZero-style MCTS planners by focusing on the information content of the MCTS structure. It introduces an information-theoretic entropy measure over MCTS subtrees and shows how to compute and update this entropy efficiently during the MCTS construction to support explanations. It then proposes entropy-guided subtree removals to balance explainability and tree size, with theoretical results on entropy change under removals and greedy algorithms evaluated in autonomous-driving simulations. The work demonstrates a practical path to generate human-understandable explanations from MCTS data and to scale explainability through controlled tree reduction.

Abstract

Complex sequential decision-making planning problems, covering infinite states' space have been shown to be solvable by AlphaZero type of algorithms. Such an approach that trains a neural model while simulating projection of futures with a Monte Carlo Tree Search algorithm were shown to be applicable to real life planning problems. As such, engineers and users interacting with the resulting policy of behavior might benefit from obtaining automated explanations about these planners' decisions offline or online. This paper focuses on the information within the Monte Carlo Tree Search data structure. Given its construction, this information contains much of the reasoning of the sequential decision-making algorithm and is essential for its explainability. We show novel methods using information theoretic tools for the simplification and reduction of the Monte Carlo Tree Search and the extraction of information. Such information can be directly used for the construction of human understandable explanations. We show that basic explainability quantities can be calculated with limited additional computational cost, as an integrated part of the Monte Carlo Tree Search construction process. We focus on the theoretical and algorithmic aspects and provide examples of how the methods presented here can be used in the construction of human understandable explanations.
Paper Structure (10 sections, 4 theorems, 40 equations, 13 figures, 4 tables, 6 algorithms)

This paper contains 10 sections, 4 theorems, 40 equations, 13 figures, 4 tables, 6 algorithms.

Table of Contents

  1. Entropy of MCTS as a Measure of Structure
  2. Reduction of MCTS using Subtree Removal
  3. Implementation & Empirical Evaluations
  4. Summary
  5. Basic Implementation of Entropy Calculation
  6. Proof of Lemma 3
  7. Proof of Theorem 1
  8. Proof of Theorem 2
  9. Reduction Implementation Additional Aspects
  10. Additional Evaluations

Key Result

Theorem 1

Assume a node at depth $i$ defined by $T_i = t$, with $N$ number of visits accumulated at its children. Its entropy is given by equation eq:entropyMCTS_recursionGeneral. When we remove child $k$ the entropy of this node is as follows: where $H_b( \cdot )$ is the binary entropy.

Figures (13)

  • Figure 1: A MCTS example
  • Figure 2: An example of the reduction using the two-stage V2 algorithm. $\beta = \frac{1}{2} \beta_{UB}$ (curve scenario).
  • Figure 3: An example of the reduction using the two-stage with criterion (second variant) algorithm. $\beta = \frac{1}{2} \beta_{UB}$ (merge scenario).
  • Figure 4: Comparing the change in trade-off in all three algorithms as a function of the factor of $\beta_{UB}$ - curve
  • Figure 5: Comparing the change in total size and entropy in all four algorithms as a function of the factor of $\beta_{UB}$ - curve
  • ...and 8 more figures

Theorems & Definitions (10)

  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Lemma 3
  • proof
  • Corollary 4
  • proof
  • proof : Proof of Theorem 1
  • proof