In Search of Trees: Decision-Tree Policy Synthesis for Black-Box Systems via Search

TL;DR

The paper addresses synthesising small, interpretable decision-tree policies for deterministic black-box systems under a bounded-time, prefix-closed specification. It introduces a backtracking search over discretised predicates combined with a novel trace-based pruning that uses observed traces to discard suboptimal branches while preserving optimality, even without access to the system model. Formal components include discretised predicates of the form $[s_i \geq v_0 + m \cdot v_+]$, a bounded-time bound $k$, and a fitness-based ordering over traces. Empirically, the authors validate substantial runtime reductions and provide insights into the effects of predicate granularity and tree size on search efficiency, demonstrating practical feasibility when small-tree policies exist and outlining future extensions to multi-start, stochastic dynamics, and parallelisation.

Abstract

Decision trees, owing to their interpretability, are attractive as control policies for (dynamical) systems. Unfortunately, constructing, or synthesising, such policies is a challenging task. Previous approaches do so by imitating a neural-network policy, approximating a tabular policy obtained via formal synthesis, employing reinforcement learning, or modelling the problem as a mixed-integer linear program. However, these works may require access to a hard-to-obtain accurate policy or a formal model of the environment (within reach of formal synthesis), and may not provide guarantees on the quality or size of the final tree policy. In contrast, we present an approach to synthesise optimal decision-tree policies given a deterministic black-box environment and specification, a discretisation of the tree predicates, and an initial set of states, where optimality is defined with respect to the number of steps to achieve the goal. Our approach is a specialised search algorithm which systematically explores the (exponentially large) space of decision trees under the given discretisation. The key component is a novel trace-based pruning mechanism that significantly reduces the search space. Our approach represents a conceptually novel way of synthesising small decision-tree policies with optimality guarantees even for black-box environments with black-box specifications.

Figures (5)

• Figure 1: Problem illustration. For the pendulum environment $E$ (right), each tree (left) with predicates $\theta\geq c_i,\:i=1,2,3,$ coupled with the black-boxed $E$, produces a trace for $\theta$ plotted along the time axis (middle) together with its evaluation with respect to the black-box specification (reaching $\theta = 0$). The middle trace is deemed best since it reaches the goal faster ($t=50<60$). The bottom trace is inferior.
• Figure 2: Three decision-tree policies of fixed structure (top) and corresponding black-box traces (bottom). The decision boundary (blue) splits the state space into two regions (light blue and red). Predicate $s_1\geq 2$ can be skipped as the trace lies above the decision boundary.
• Figure 3: Visualisation of example decision trees for depth two, predicate increments $\{0.1,0.1,0.1,0.1\}$, and random with seed $0$ initial state $[0.013, -0.02, 0.047, 0.025]$ (left) and $100$ randomly sampled initial states (right) with seed $9$.
• Figure 4: Visualisation of example decision trees for depth two, predicate increments $\{0.05,0.005\}$, and random with seed $0$ initial state $[-0.51,0]$ (left) and $100$ randomly sampled initial states (right) with seed $9$.
• Figure 5: Visualisation of example decision trees for depth two, predicate increments $\{0.2,0.2\}$, and random with seed $0$ initial state $[-0.665, -0.024]$ (left) and $100$ randomly sampled initial states (right) with seed $9$.

Theorems & Definitions (1)

• Example 1