Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HAVEN: Hierarchical Adversary-aware Visibility-Enabled Navigation with Cover Utilization using Deep Transformer Q-Networks

Mihir Chauhan, Damon Conover, Aniket Bera

TL;DR

This work tackles safe navigation under partial observability and adversarial visibility by introducing HAVEN, a hierarchical framework that uses a high-level Deep Transformer Q-Network for subgoal selection and a low-level potential-field controller for reactive execution. The approach employs visibility-aware candidate generation and a compact 16-D feature encoding to reason about occlusion, cover, and adversarial FoVs, enabling memory-enabled decision making. HAVEN demonstrates transfer from 2D to 3D Unity-ROS environments without architectural changes, by projecting perception onto a common feature schema. Empirical results show superior success rates, safety margins, and reduced adversarial exposure compared to classical planners and RL baselines, with ablations confirming the value of temporal memory and visibility design for navigation under uncertainty.

Abstract

Autonomous navigation in partially observable environments requires agents to reason beyond immediate sensor input, exploit occlusion, and ensure safety while progressing toward a goal. These challenges arise in many robotics domains, from urban driving and warehouse automation to defense and surveillance. Classical path planning approaches and memoryless reinforcement learning often fail under limited fields of view (FoVs) and occlusions, committing to unsafe or inefficient maneuvers. We propose a hierarchical navigation framework that integrates a Deep Transformer Q-Network (DTQN) as a high-level subgoal selector with a modular low-level controller for waypoint execution. The DTQN consumes short histories of task-aware features, encoding odometry, goal direction, obstacle proximity, and visibility cues, and outputs Q-values to rank candidate subgoals. Visibility-aware candidate generation introduces masking and exposure penalties, rewarding the use of cover and anticipatory safety. A low-level potential field controller then tracks the selected subgoal, ensuring smooth short-horizon obstacle avoidance. We validate our approach in 2D simulation and extend it directly to a 3D Unity-ROS environment by projecting point-cloud perception into the same feature schema, enabling transfer without architectural changes. Results show consistent improvements over classical planners and RL baselines in success rate, safety margins, and time to goal, with ablations confirming the value of temporal memory and visibility-aware candidate design. These findings highlight a generalizable framework for safe navigation under uncertainty, with broad relevance across robotic platforms.

HAVEN: Hierarchical Adversary-aware Visibility-Enabled Navigation with Cover Utilization using Deep Transformer Q-Networks

TL;DR

This work tackles safe navigation under partial observability and adversarial visibility by introducing HAVEN, a hierarchical framework that uses a high-level Deep Transformer Q-Network for subgoal selection and a low-level potential-field controller for reactive execution. The approach employs visibility-aware candidate generation and a compact 16-D feature encoding to reason about occlusion, cover, and adversarial FoVs, enabling memory-enabled decision making. HAVEN demonstrates transfer from 2D to 3D Unity-ROS environments without architectural changes, by projecting perception onto a common feature schema. Empirical results show superior success rates, safety margins, and reduced adversarial exposure compared to classical planners and RL baselines, with ablations confirming the value of temporal memory and visibility design for navigation under uncertainty.

Abstract

Autonomous navigation in partially observable environments requires agents to reason beyond immediate sensor input, exploit occlusion, and ensure safety while progressing toward a goal. These challenges arise in many robotics domains, from urban driving and warehouse automation to defense and surveillance. Classical path planning approaches and memoryless reinforcement learning often fail under limited fields of view (FoVs) and occlusions, committing to unsafe or inefficient maneuvers. We propose a hierarchical navigation framework that integrates a Deep Transformer Q-Network (DTQN) as a high-level subgoal selector with a modular low-level controller for waypoint execution. The DTQN consumes short histories of task-aware features, encoding odometry, goal direction, obstacle proximity, and visibility cues, and outputs Q-values to rank candidate subgoals. Visibility-aware candidate generation introduces masking and exposure penalties, rewarding the use of cover and anticipatory safety. A low-level potential field controller then tracks the selected subgoal, ensuring smooth short-horizon obstacle avoidance. We validate our approach in 2D simulation and extend it directly to a 3D Unity-ROS environment by projecting point-cloud perception into the same feature schema, enabling transfer without architectural changes. Results show consistent improvements over classical planners and RL baselines in success rate, safety margins, and time to goal, with ablations confirming the value of temporal memory and visibility-aware candidate design. These findings highlight a generalizable framework for safe navigation under uncertainty, with broad relevance across robotic platforms.

Paper Structure

This paper contains 38 sections, 16 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Approaches for Collision Avoidance
  4. Adversarial Threat-Aware Navigation
  5. Hierarchical Reinforcement Learning Control Strategies
  6. Deep Transformer Q-Network
  7. Problem
  8. State Space
  9. Observation Model
  10. Reward Function
  11. Candidate Subgoal Features
  12. Evaluation Metrics
  13. Methodology
  14. Hierarchical Policy Structure
  15. Observation Model
  16. ...and 23 more sections

Figures (4)

  • Figure 1: Proposed hierarchical architecture for enemy-aware path planning with cover utilization using high-level Deep Transformer Q-Network and low-level potential field controller.
  • Figure 2: Proposed hierarchical architecture. The high-level DTQN outputs Q-values for candidate subgoals, and the selected subgoal is passed to the low-level potential field controller for execution.
  • Figure 3: 2D training and evaluation setup with 5 enemy agents (red) and the navigating agent (blue). The agent’s trajectory avoids exposure by using cover (obstacle circled in pink) before reaching the goal (grey square at (15,15)). Enemy spawn points and obstacles are randomized at the start of each run.
  • Figure 4: 3D evaluation pipeline. A 2D-trained DTQN + low-level controller are deployed in Unity--ROS: LiDAR point clouds are projected to the ground plane, denoised via occupancy/persistence, polygonized, and fed to the 2D feature schema. Selected subgoals are executed as linear/angular velocity commands in ROS. The red circle depicts an adversarial threat in the point cloud.