Aerial Gym Simulator: A Framework for Highly Parallelized Simulation of Aerial Robots

TL;DR

The paper introduces the Aerial Gym Simulator, a GPU-accelerated, highly parallel framework built on NVIDIA Isaac Gym that enables simulation of arbitrary multirotor configurations with rich exteroceptive sensing. It combines a modular architecture, a comprehensive controller suite, and a custom GPU-based rendering pipeline for depth, segmentation, and normals, along with exteroceptive sensor models and runnable DRL environments compatible with Gymnasium. Key contributions include support for arbitrary motor counts and reconfigurable airframes, multiple control abstractions, a pseudoinverse-based body-wrench allocation, and a scalable, ray-casting rendering framework that significantly accelerates sensor simulation. The authors validate sim2real transfer for state-based and vision-based navigation policies and provide open-source access to the toolkit, enabling researchers to train and deploy policies for learning-based aerial control and planning with exteroceptive data.

Abstract

This paper contributes the Aerial Gym Simulator, a highly parallelized, modular framework for simulation and rendering of arbitrary multirotor platforms based on NVIDIA Isaac Gym. Aerial Gym supports the simulation of under-, fully- and over-actuated multirotors offering parallelized geometric controllers, alongside a custom GPU-accelerated rendering framework for ray-casting capable of capturing depth, segmentation and vertex-level annotations from the environment. Multiple examples for key tasks, such as depth-based navigation through reinforcement learning are provided. The comprehensive set of tools developed within the framework makes it a powerful resource for research on learning for control, planning, and navigation using state information as well as exteroceptive sensor observations. Extensive simulation studies are conducted and successful sim2real transfer of trained policies is demonstrated. The Aerial Gym Simulator is open-sourced at: https://github.com/ntnu-arl/aerial_gym_simulator.

Figures (7)

• Figure 1: Salient features of the Aerial Gym Simulator.
• Figure 2: Core components for the Aerial Gym Simulator include the physics engine (Isaac Gym), Warp and Isaac Gym-based rendering solutions, robot control suite and tools for environment generation and randomization.
• Figure 3: Various airframes provided with the Aerial Gym Simulator. Rigid quad- (a) and octa-rotors (b and c), alongside a compliant robot Morphy (d) based on paolo2024morphy and reconfigurable robot (e) inspired by zhao2017whole.
• Figure 4: Sensor measurements using the proposed rendering framework. \ref{['subfig:camera_sensor']} shows depth, segmentation, surface-normal and face-index images captured by a simulated camera sensor. \ref{['subfig:lidar_sensor']} shows the data captured using a standard $3$-D LiDAR and a hemispherical dome LiDAR sensor.
• Figure 5: Real-world testing of position-setpoint tracking policies with velocity (left) and acceleration (right) commands. Solid lines show data from the real-world experiment, while the dashed lines and the shaded regions indicate mean and standard deviation across policies evaluated in simulation.