SERN: Simulation-Enhanced Realistic Navigation for Multi-Agent Robotic Systems in Contested Environments

TL;DR

The paper tackles cross-domain coordination for multi-agent robotics in contested environments by bridging physical and virtual domains through SERN, a Simulation-Enhanced Realistic Navigation framework. It combines a bi-directional ROS Bridge on AuroraXR with a Unity-based virtual world, physics-aware synchronization, and a Multi-Metric Cost Function to optimize bridge configurations under variable network conditions. Key contributions include adaptive virtual environment fidelity (LoD), semantic data management to reduce bandwidth, a physics-driven synchronization mechanism yielding sub-centimeter positional accuracy and sub-degree rotations during disruptions, and demonstrated latency reductions (15–24%) with improved processing efficiency. These advances enable scalable, real-time situational awareness and coordinated decision-making across distributed robotic teams in challenging environments, with practical impact on mission planning and remote operation.

Abstract

The increasing deployment of autonomous systems in complex environments necessitates efficient communication and task completion among multiple agents. This paper presents SERN (Simulation-Enhanced Realistic Navigation), a novel framework integrating virtual and physical environments for real-time collaborative decision-making in multi-robot systems. SERN addresses key challenges in asset deployment and coordination through our bi-directional SERN ROS Bridge communication framework. Our approach advances the SOTA through: accurate real-world representation in virtual environments using Unity high-fidelity simulator; synchronization of physical and virtual robot movements; efficient ROS data distribution between remote locations; and integration of SOTA semantic segmentation for enhanced environmental perception. Additionally, we introduce a Multi-Metric Cost Function (MMCF) that dynamically balances latency, reliability, computational overhead, and bandwidth consumption to optimize system performance in contested environments. We further provide theoretical justification for synchronization accuracy by proving that the positional error between physical and virtual robots remains bounded under varying network conditions. Our evaluations show a 15% to 24% improvement in latency and up to a 15% increase in processing efficiency compared to traditional ROS setups. Real-world and virtual simulation experiments with multiple robots (Clearpath Jackal and Husky) demonstrate synchronization accuracy, achieving less than $5\text{ cm}$ positional error and under $2^\circ$ rotational error. These results highlight SERN's potential to enhance situational awareness and multi-agent coordination in diverse, contested environments.

Figures (6)

• Figure 1: SERN framework implementation demonstrating bi-directional communication between physical and virtual robots. The system enables synchronized path planning and cross-domain data sharing across separate physical and virtual networks, supporting coordinated multi-robot operations.
• Figure 2: SERN: Bi-directional Communication Framework Overview
• Figure 3: Visualization of data received from SERN framework.
• Figure 4: SERN ROS Bridge setup and network diagram. (a) A high-level view of the network architecture illustrating how robots connect through an access point to the AuroraXR server, enabling real-time data exchange. (b) The actual hardware testbed, showing multiple laptops (each functioning as an individual robot) running the AuroraXR environment alongside ROS nodes for coordinated robot control.
• Figure 5: Virtual and physical environments used for evaluating SERN framework. (a) Virtual environment showing the simulated robot navigating an area with obstacles (b) Corresponding physical environment with the real robot executing navigation tasks in a similar setup, demonstrating real-world implementation and validation of the framework. (c) Semantic segmentation output from the environment, highlighting the system’s ability to identify key features such as obstacles and navigable paths, ensuring synchronized perception and navigation between the virtual and physical domains.

Theorems & Definitions (2)

• Proposition 1
• proof