FH-DRL: Exponential-Hyperbolic Frontier Heuristics with DRL for accelerated Exploration in Unknown Environments

TL;DR

FH-DRL presents a hybrid frontier exploration framework that unites a novel frontier heuristic with a TD3-based local navigator. By coupling an Exponential-Hyperbolic Distance Score with an Occupancy Stochastic Score, the method prioritizes informative frontiers while accounting for openness and obstacle density, and it integrates TD3-based control for high-speed, robust navigation. Across simulations and real-world Turtlebot3 experiments, FH-DRL consistently reduces travel distance and completion time while achieving high exploration rates compared to frontier-only and pure DRL approaches. The approach demonstrates strong performance in structured corridors and maze-like environments, offering a scalable solution for autonomous exploration in partially known or dynamic environments with broad applicability in robotics.

Abstract

Autonomous robot exploration in large-scale or cluttered environments remains a central challenge in intelligent vehicle applications, where partial or absent prior maps constrain reliable navigation. This paper introduces FH-DRL, a novel framework that integrates a customizable heuristic function for frontier detection with a Twin Delayed DDPG (TD3) agent for continuous, high-speed local navigation. The proposed heuristic relies on an exponential-hyperbolic distance score, which balances immediate proximity against long-range exploration gains, and an occupancy-based stochastic measure, accounting for environmental openness and obstacle densities in real time. By ranking frontiers using these adaptive metrics, FH-DRL targets highly informative yet tractable waypoints, thereby minimizing redundant paths and total exploration time. We thoroughly evaluate FH-DRL across multiple simulated and real-world scenarios, demonstrating clear improvements in travel distance and completion time over frontier-only or purely DRL-based exploration. In structured corridor layouts and maze-like topologies, our architecture consistently outperforms standard methods such as Nearest Frontier, Cognet Frontier Exploration, and Goal Driven Autonomous Exploration. Real-world tests with a Turtlebot3 platform further confirm robust adaptation to previously unseen or cluttered indoor spaces. The results highlight FH-DRL as an efficient and generalizable approach for frontier-based exploration in large or partially known environments, offering a promising direction for various autonomous driving, industrial, and service robotics tasks.

Figures (11)

• Figure 1: Overall Robot Navigation for Exploration Architecture
• Figure 2: Exponential-Hyperbolic Distance Score
• Figure 3: Visualisation of exponential-hyperbolic distance and occupancy grid score distribusion
• Figure 4: Representative Frontiers (a). Closed Frontier, (b). Open Wided Frontiers, (c). Door Gap Frontier, (d) Case-study
• Figure 5: FH-DRL score 3D Distribution