Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hierarchical Neuro-Symbolic Decision Transformer

Ali Baheri, Cecilia O. Alm

TL;DR

The paper addresses the challenge of long-horizon decision-making under uncertainty by uniting high-level symbolic planning with a transformer-based low-level policy. It introduces a hierarchical neuro-symbolic decision transformer that uses a bidirectional interface to translate symbolic operators into sub-goals for a decision transformer and to map raw states back into symbolic predicates, preserving interpretability while leveraging deep sequence modeling. Theoretical results provide hierarchical regret and PAC bounds that quantify how planning and execution errors propagate through the system. Empirical evaluation on stochastic grid-worlds demonstrates improved success rates and sample efficiency over purely symbolic, purely neural, and existing hierarchical baselines, with notable robustness to action-noise and task complexity. The approach offers a principled framework for scalable, interpretable, and resilient sequential decision-making with potential extensions to continuous domains and robotics.

Abstract

We present a hierarchical neuro-symbolic control framework that tightly couples a classical symbolic planner with a transformer-based policy to address long-horizon decision-making under uncertainty. At the high level, the planner assembles an interpretable sequence of operators that guarantees logical coherence with task constraints, while at the low level each operator is rendered as a sub-goal token that conditions a decision transformer to generate fine-grained actions directly from raw observations. This bidirectional interface preserves the combinatorial efficiency and explainability of symbolic reasoning without sacrificing the adaptability of deep sequence models, and it permits a principled analysis that tracks how approximation errors from both planning and execution accumulate across the hierarchy. Empirical studies in stochastic grid-world domains demonstrate that the proposed method consistently surpasses purely symbolic, purely neural and existing hierarchical baselines in both success and efficiency, highlighting its robustness for sequential tasks.

Hierarchical Neuro-Symbolic Decision Transformer

TL;DR

The paper addresses the challenge of long-horizon decision-making under uncertainty by uniting high-level symbolic planning with a transformer-based low-level policy. It introduces a hierarchical neuro-symbolic decision transformer that uses a bidirectional interface to translate symbolic operators into sub-goals for a decision transformer and to map raw states back into symbolic predicates, preserving interpretability while leveraging deep sequence modeling. Theoretical results provide hierarchical regret and PAC bounds that quantify how planning and execution errors propagate through the system. Empirical evaluation on stochastic grid-worlds demonstrates improved success rates and sample efficiency over purely symbolic, purely neural, and existing hierarchical baselines, with notable robustness to action-noise and task complexity. The approach offers a principled framework for scalable, interpretable, and resilient sequential decision-making with potential extensions to continuous domains and robotics.

Abstract

We present a hierarchical neuro-symbolic control framework that tightly couples a classical symbolic planner with a transformer-based policy to address long-horizon decision-making under uncertainty. At the high level, the planner assembles an interpretable sequence of operators that guarantees logical coherence with task constraints, while at the low level each operator is rendered as a sub-goal token that conditions a decision transformer to generate fine-grained actions directly from raw observations. This bidirectional interface preserves the combinatorial efficiency and explainability of symbolic reasoning without sacrificing the adaptability of deep sequence models, and it permits a principled analysis that tracks how approximation errors from both planning and execution accumulate across the hierarchy. Empirical studies in stochastic grid-world domains demonstrate that the proposed method consistently surpasses purely symbolic, purely neural and existing hierarchical baselines in both success and efficiency, highlighting its robustness for sequential tasks.

Paper Structure

This paper contains 35 sections, 2 theorems, 26 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Symbolic Planning.
  4. Hierarchical Reinforcement Learning.
  5. Transformer-Based Reinforcement Learning.
  6. Neuro-Symbolic Approaches.
  7. Hybrid Planning and Learning Frameworks.
  8. Hierarchical Modeling and Model-based Reinforcement Learning.
  9. Preliminaries: Decision Transformers
  10. Methodology
  11. Theoretical Results
  12. Numerical Results
  13. Conclusion
  14. Detailed Environment Models for the Case Studies
  15. Case Study 1: Single Key--Door Model
  16. ...and 20 more sections

Key Result

theorem 1

Let $0<\gamma<1$ be the discount factor of the underlying MDP, and let $V^{\!*}$ denote the optimal value function. Suppose the symbolic planner produces a plan whose expected cost is at most $\epsilon_{\text{sym}}$ above the optimal symbolic cost. Assume further that the decision transformer execut where $C_i^{\!*}$ is the optimal cost for realizing $o_i$. Let $\rho$ be an upper bound on the prob

Figures (6)

  • Figure 1: Hierarchical Neuro-Symbolic Control Framework
  • Figure 2: Case Study 1 - Single Key-Door Environment
  • Figure 3: Case Study 2 - Multi-Goal Key-Door Environment
  • Figure 4: Comprehensive performance comparison across both environments and varying action failure probabilities. Our method consistently achieves superior performance.
  • Figure 5: Performance comparison categorized by architectural paradigm, highlighting the advantages of the neuro-symbolic approach.
  • ...and 1 more figures

Theorems & Definitions (4)

  • theorem 1: Hierarchical Performance Bound
  • proof : Sketch
  • theorem 2: PAC Sub‑goal Completion
  • proof : Sketch