Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DRL-TH: Jointly Utilizing Temporal Graph Attention and Hierarchical Fusion for UGV Navigation in Crowded Environments

Ruitong Li, Lin Zhang, Yuenan Zhao, Chengxin Liu, Ran Song, Wei Zhang

TL;DR

This work tackles autonomous UGV navigation in crowded environments where single-frame perception and simple fusion limit performance. It introduces DRL-TH, a framework that jointly leverages a temporal-guided graph attention network (TG-GAT) and a graph hierarchical abstraction module (GHAM) to incorporate historical observations and adaptively fuse RGB and LiDAR features, guided by a PPO-based policy. The approach achieves superior navigation performance in simulated CARLA scenes and real-world tests, showing robustness to obstacle density, weather, and real-world transfer without fine-tuning. The results highlight the value of explicit temporal reasoning and multi-scale, learnable fusion for reliable UGV operation in dynamic, cluttered environments.

Abstract

Deep reinforcement learning (DRL) methods have demonstrated potential for autonomous navigation and obstacle avoidance of unmanned ground vehicles (UGVs) in crowded environments. Most existing approaches rely on single-frame observation and employ simple concatenation for multi-modal fusion, which limits their ability to capture temporal context and hinders dynamic adaptability. To address these challenges, we propose a DRL-based navigation framework, DRL-TH, which leverages temporal graph attention and hierarchical graph pooling to integrate historical observations and adaptively fuse multi-modal information. Specifically, we introduce a temporal-guided graph attention network (TG-GAT) that incorporates temporal weights into attention scores to capture correlations between consecutive frames, thereby enabling the implicit estimation of scene evolution. In addition, we design a graph hierarchical abstraction module (GHAM) that applies hierarchical pooling and learnable weighted fusion to dynamically integrate RGB and LiDAR features, achieving balanced representation across multiple scales. Extensive experiments demonstrate that our DRL-TH outperforms existing methods in various crowded environments. We also implemented DRL-TH control policy on a real UGV and showed that it performed well in real world scenarios.

DRL-TH: Jointly Utilizing Temporal Graph Attention and Hierarchical Fusion for UGV Navigation in Crowded Environments

TL;DR

This work tackles autonomous UGV navigation in crowded environments where single-frame perception and simple fusion limit performance. It introduces DRL-TH, a framework that jointly leverages a temporal-guided graph attention network (TG-GAT) and a graph hierarchical abstraction module (GHAM) to incorporate historical observations and adaptively fuse RGB and LiDAR features, guided by a PPO-based policy. The approach achieves superior navigation performance in simulated CARLA scenes and real-world tests, showing robustness to obstacle density, weather, and real-world transfer without fine-tuning. The results highlight the value of explicit temporal reasoning and multi-scale, learnable fusion for reliable UGV operation in dynamic, cluttered environments.

Abstract

Deep reinforcement learning (DRL) methods have demonstrated potential for autonomous navigation and obstacle avoidance of unmanned ground vehicles (UGVs) in crowded environments. Most existing approaches rely on single-frame observation and employ simple concatenation for multi-modal fusion, which limits their ability to capture temporal context and hinders dynamic adaptability. To address these challenges, we propose a DRL-based navigation framework, DRL-TH, which leverages temporal graph attention and hierarchical graph pooling to integrate historical observations and adaptively fuse multi-modal information. Specifically, we introduce a temporal-guided graph attention network (TG-GAT) that incorporates temporal weights into attention scores to capture correlations between consecutive frames, thereby enabling the implicit estimation of scene evolution. In addition, we design a graph hierarchical abstraction module (GHAM) that applies hierarchical pooling and learnable weighted fusion to dynamically integrate RGB and LiDAR features, achieving balanced representation across multiple scales. Extensive experiments demonstrate that our DRL-TH outperforms existing methods in various crowded environments. We also implemented DRL-TH control policy on a real UGV and showed that it performed well in real world scenarios.
Paper Structure (27 sections, 24 equations, 8 figures, 4 tables)

This paper contains 27 sections, 24 equations, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reinforcement Learning for Navigation
  4. Graph Neural Networks for Navigation
  5. Method
  6. Reinforcement Learning for UGV Navigation
  7. Algorithm Selection
  8. Observation Space and Action Space
  9. Reward Function
  10. Network Architecture
  11. Temporal-Guided Graph Attention Network for Implicit Context Reasoning
  12. Graph Hierarchical Abstraction Module for Multi-modal Fusion
  13. Hierarchical Pooling
  14. Weighted Fusion
  15. Experiments
  16. ...and 12 more sections

Figures (8)

  • Figure 1: Overall of our DRL-TH framework, where the detailed architectures of the temporal-guided graph attention network (TG-GAT) and graph hierarchical abstraction module (GHAM) are illustrated in Figs. \ref{['TGGAT']} and \ref{['diffpool']}.
  • Figure 2: Illustration of TG-GAT, incorporating temporal attention weights into the graph attention network framework.
  • Figure 3: Structure of the GHAM. Hierarchical pooling reduces the graph from 6 to 2, then to 1 node, with adaptive weighted fusion dynamically balancing modality features across scales.
  • Figure 4: The UGV and map used in the real-world test, where the main external sensors and configuration are annotated.
  • Figure 5: Average reward in two-stage training scenarios. (a) conventional stage. (b) crowded stage.
  • ...and 3 more figures