Unreal Robotics Lab: A High-Fidelity Robotics Simulator with Advanced Physics and Rendering
Jonathan Embley-Riches, Jianwei Liu, Simon Julier, Dimitrios Kanoulas
TL;DR
The paper tackles the sim-to-real gap by integrating Unreal Engine's photorealistic rendering with MuJoCo's accurate physics to create a general-purpose, high-fidelity robotics simulator. It introduces Unreal Robotics Lab (URL), an open-source ROS-enabled framework that automates scene construction, physics synchronization, sensor/actuator interfaces, benchmarking, and replay, while supporting adverse environmental effects via Niagara. Through experiments on visual navigation and SLAM under diverse adversity and real-world comparisons, the authors show realistic perception and reveal brittleness of current vision systems, underscoring the value of synthetic, controllable data for robustness and sim-to-real transfer. By supporting quadrupeds, manipulators, UAVs, and more, URL provides a scalable platform for perception training, benchmark generation, and system-level robotics validation.
Abstract
High-fidelity simulation is essential for robotics research, enabling safe and efficient testing of perception, control, and navigation algorithms. However, achieving both photorealistic rendering and accurate physics modeling remains a challenge. This paper presents a novel simulation framework, the Unreal Robotics Lab (URL), that integrates the advanced rendering capabilities of the Unreal Engine with MuJoCo's high-precision physics simulation. Our approach enables realistic robotic perception while maintaining accurate physical interactions, facilitating benchmarking and dataset generation for vision-based robotics applications. The system supports complex environmental effects, such as smoke, fire, and water dynamics, which are critical to evaluating robotic performance under adverse conditions. We benchmark visual navigation and SLAM methods within our framework, demonstrating its utility for testing real-world robustness in controlled yet diverse scenarios. By bridging the gap between physics accuracy and photorealistic rendering, our framework provides a powerful tool for advancing robotics research and sim-to-real transfer. Our open-source framework is available at https://unrealroboticslab.github.io/.