EnCoMP: Enhanced Covert Maneuver Planning with Adaptive Threat-Aware Visibility Estimation using Offline Reinforcement Learning

TL;DR

EnCoMP tackles covert outdoor navigation by learning a robust policy from real-world LiDAR data using offline reinforcement learning (Conservative Q-Learning) and by integrating multi-map perception (height, cover, threat, and goal maps). The Adaptive Threat-Aware Visibility Estimation (ATAVE) dynamically models threats and guides planning to minimize exposure while maximizing cover usage. Experimental results on a Jackal robot across urban, forest, and mixed terrains show superior success rates, reduced threat exposure, and higher cover utilization compared with state-of-the-art baselines, with real-time decision speed ($0.05$–$0.08$ s). The work demonstrates the practical viability of offline-learned covert navigation with real-time threat adaptation, enabling safer and more efficient stealthy missions in complex outdoor environments.

Abstract

Autonomous robots operating in complex environments face the critical challenge of identifying and utilizing environmental cover for covert navigation to minimize exposure to potential threats. We propose EnCoMP, an enhanced navigation framework that integrates offline reinforcement learning and our novel Adaptive Threat-Aware Visibility Estimation (ATAVE) algorithm to enable robots to navigate covertly and efficiently in diverse outdoor settings. ATAVE is a dynamic probabilistic threat modeling technique that we designed to continuously assess and mitigate potential threats in real-time, enhancing the robot's ability to navigate covertly by adapting to evolving environmental and threat conditions. Moreover, our approach generates high-fidelity multi-map representations, including cover maps, potential threat maps, height maps, and goal maps from LiDAR point clouds, providing a comprehensive understanding of the environment. These multi-maps offer detailed environmental insights, helping in strategic navigation decisions. The goal map encodes the relative distance and direction to the target location, guiding the robot's navigation. We train a Conservative Q-Learning (CQL) model on a large-scale dataset collected from real-world environments, learning a robust policy that maximizes cover utilization, minimizes threat exposure, and maintains efficient navigation. We demonstrate our method's capabilities on a physical Jackal robot, showing extensive experiments across diverse terrains. These experiments demonstrate EnCoMP's superior performance compared to state-of-the-art methods, achieving a 95% success rate, 85% cover utilization, and reducing threat exposure to 10.5%, while significantly outperforming baselines in navigation efficiency and robustness.

Figures (5)

• Figure 1: EnCoMP Covert Navigation Strategies: In this outdoor environment, the robot (Jackal) is navigating towards a goal location. To achieve this while minimizing the risk of detection or interference from potential threats—defined as any external factors capable of recognizing or interfering with the robot's presence and objectives (represented by the eye icons)—the robot identifies and strategically moves toward nearby trees and a small brick building structure that can provide cover and concealment, instead of taking the risky route through the open area. By utilizing these natural and artificial features in the environment, the robot reduces its visibility and exposure as it traverses the field to reach its destination safely.
• Figure 2: Overview of EnCoMP System Architecture.
• Figure 3: Visualization of the LLA* Visibility Calculation in the EnCoMP framework. The plot shows the grid environment with obstacles (gray cells), the robot's position (green circle), the threat's position (red cross), and the visible cells (orange dots) determined by the Late Line-of-Sight Check and Prioritized Trees (LLA*) algorithm. The red dashed lines represent the line-of-sight from the threat to the visible cells within its visibility range.
• Figure 4: Comparison of navigation strategies in diverse outdoor settings (See \ref{['sec:testingscenario']}). (a) Scenario 1, (b) Scenario 2, (c) Scenario 3. Paths indicate the trajectories taken by different systems, with EnCoMP (blue) demonstrating more optimized routes that effectively leverage cover while minimizing threat exposure, in contrast to CoverNav (green), VAPOR (purple), and VERN (yellow).
• Figure 5: A visualization of the threat map and the trajectories generated by EnCoMP and CoverNav hossain2023covernav in a representative scenario. The threat map, depicted by the colormap, indicates the level of threat associated with different locations in the environment, with darker colors representing higher threat levels. The EnCoMP trajectory (orange line with circular markers) navigates through regions with lower threat levels, demonstrating its ability to identify and prioritize safer routes. In contrast, the CoverNav trajectory (blue line with square markers) traverses through areas with higher threat levels, suggesting a less effective threat avoidance capability. This visualization highlights EnCoMP's superior performance in generating safer paths and minimizing exposure to high-threat regions compared to CoverNav.