Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Traversal-as-Policy: Log-Distilled Gated Behavior Trees as Externalized, Verifiable Policies for Safe, Robust, and Efficient Agents

Peiran Li, Jiashuo Sun, Fangzhou Lin, Shuo Xing, Tianfu Fu, Suofei Feng, Chaoqun Ni, Zhengzhong Tu

TL;DR

GBT improves success while driving violations toward zero and reducing cost in a unified OpenHands sandbox, and is evaluated in a unified OpenHands sandbox on 15+ software, web, reasoning, and safety/security benchmarks.

Abstract

Autonomous LLM agents fail because long-horizon policy remains implicit in model weights and transcripts, while safety is retrofitted post hoc. We propose Traversal-as-Policy: distill sandboxed OpenHands execution logs into a single executable Gated Behavior Tree (GBT) and treat tree traversal -- rather than unconstrained generation -- as the control policy whenever a task is in coverage. Each node encodes a state-conditioned action macro mined and merge-checked from successful trajectories; macros implicated by unsafe traces attach deterministic pre-execution gates over structured tool context and bounded history, updated under experience-grounded monotonicity so previously rejected unsafe contexts cannot be re-admitted. At runtime, a lightweight traverser matches the base model's intent to child macros, executes one macro at a time under global and node-local gating, and when stalled performs risk-aware shortest-path recovery to a feasible success leaf; the visited path forms a compact spine memory that replaces transcript replay. Evaluated in a unified OpenHands sandbox on 15+ software, web, reasoning, and safety/security benchmarks, GBT improves success while driving violations toward zero and reducing cost. On SWE-bench Verified (Protocol A, 500 issues), GBT-SE raises success from 34.6% to 73.6%, reduces violations from 2.8% to 0.2%, and cuts token/character usage from 208k/820k to 126k/490k; with the same distilled tree, 8B executors more than double success on SWE-bench Verified (14.0%58.8%) and WebArena (9.1%37.3%).

Traversal-as-Policy: Log-Distilled Gated Behavior Trees as Externalized, Verifiable Policies for Safe, Robust, and Efficient Agents

TL;DR

GBT improves success while driving violations toward zero and reducing cost in a unified OpenHands sandbox, and is evaluated in a unified OpenHands sandbox on 15+ software, web, reasoning, and safety/security benchmarks.

Abstract

Autonomous LLM agents fail because long-horizon policy remains implicit in model weights and transcripts, while safety is retrofitted post hoc. We propose Traversal-as-Policy: distill sandboxed OpenHands execution logs into a single executable Gated Behavior Tree (GBT) and treat tree traversal -- rather than unconstrained generation -- as the control policy whenever a task is in coverage. Each node encodes a state-conditioned action macro mined and merge-checked from successful trajectories; macros implicated by unsafe traces attach deterministic pre-execution gates over structured tool context and bounded history, updated under experience-grounded monotonicity so previously rejected unsafe contexts cannot be re-admitted. At runtime, a lightweight traverser matches the base model's intent to child macros, executes one macro at a time under global and node-local gating, and when stalled performs risk-aware shortest-path recovery to a feasible success leaf; the visited path forms a compact spine memory that replaces transcript replay. Evaluated in a unified OpenHands sandbox on 15+ software, web, reasoning, and safety/security benchmarks, GBT improves success while driving violations toward zero and reducing cost. On SWE-bench Verified (Protocol A, 500 issues), GBT-SE raises success from 34.6% to 73.6%, reduces violations from 2.8% to 0.2%, and cuts token/character usage from 208k/820k to 126k/490k; with the same distilled tree, 8B executors more than double success on SWE-bench Verified (14.0%58.8%) and WebArena (9.1%37.3%).
Paper Structure (129 sections, 22 equations, 1 figure, 23 tables)

This paper contains 129 sections, 22 equations, 1 figure, 23 tables.

Table of Contents

  1. Introduction
  2. Gated Behavior Tree (GBT)
  3. Safety Objects and Experience-Grounded Gates
  4. Global Guardrail
  5. From Logs to GBT-Basic
  6. Self-Evolution Under Safety Invariants: GBT-SE
  7. Online Traversal-as-Policy
  8. Policy as an External Artifact
  9. Experiments
  10. Reproducible Runtime and Integration Discipline
  11. Benchmarks, Protocols, and Metrics
  12. Main Results on the Three Pillars (Protocol A)
  13. SWE-bench Verified: large utility gains without safety trade-offs
  14. WebArena: traversal arrests drift; recovery converts stalls into progress
  15. GPQA: with browsing disabled, GBT turns the agent into a controlled local executor
  16. ...and 114 more sections

Figures (1)

  • Figure 1: Overview of Traversal-as-Policy. Gated Behavior Tree (GBT) distills offline logs into a gated tree artifact. Online traversal wraps a base LLM, executing macros under deterministic gates and using risk-aware recovery. Self-evolution safely evolves the policy under strict regression invariants.