OmniNxt: A Fully Open-source and Compact Aerial Robot with Omnidirectional Visual Perception

TL;DR

OmniNxt tackles the lack of open-source, compact platforms with omnidirectional perception for aerial robots. It combines a coin-sized, PX4-based flight controller (Nxt-FC), a multi-fisheye camera set, and an onboard Nvidia Jetson Orin NX to enable real-time omnidirectional perception through two core modules: Omni-VINS for visual-inertial odometry and Omni-Depth for dense mapping. The system is designed around the COPE criteria (Compact, Open-source, Perceptive, Extendable) and validated with real-world experiments showing improved localization accuracy and robust dense mapping, along with autonomous indoor navigation capabilities. The authors provide open hardware and software resources, including flexible sensor adaptations, making OmniNxt a practical platform for researchers and developers advancing omnidirectional aerial perception.

Abstract

Adopting omnidirectional Field of View (FoV) cameras in aerial robots vastly improves perception ability, significantly advancing aerial robotics's capabilities in inspection, reconstruction, and rescue tasks. However, such sensors also elevate system complexity, e.g., hardware design, and corresponding algorithm, which limits researchers from utilizing aerial robots with omnidirectional FoV in their research. To bridge this gap, we propose OmniNxt, a fully open-source aerial robotics platform with omnidirectional perception. We design a high-performance flight controller NxtPX4 and a multi-fisheye camera set for OmniNxt. Meanwhile, the compatible software is carefully devised, which empowers OmniNxt to achieve accurate localization and real-time dense mapping with limited computation resource occupancy. We conducted extensive real-world experiments to validate the superior performance of OmniNxt in practical applications. All the hardware and software are open-access at https://github.com/HKUST-Aerial-Robotics/OmniNxt, and we provide docker images of each crucial module in the proposed system. Project page: https://hkust-aerial-robotics.github.io/OmniNxt.

Figures (10)

• Figure 1: The system overview of OmniNxt. The hardware architecture includes: Multi-fisheye camera set (camera), Nxt-FC (flight controller and IMU), and Nvidia Jeston Orin NX (onboard computation). The software framework consists of two critical components: Omni-VINS (Sec. \ref{['subsec:omnidirectional_vins']}) and Omni-Depth (Sec. \ref{['subsec:omnidirectional_depth_estimation']}).
• Figure 2: Typical structures of omnidirectional FoV. Cameras in the UpDown structure are on the top and bottom of the platform, facing upwards and downwards. The Corner structure places cameras on the corners of a plane, covering an omnidirectional view.
• Figure 3: Platforms comparison. The platforms are compared based on their FoV and the ratio of onboard computation power to the product of size and weight. A higher value on the vertical axis indicates the platform's ability to perceive the surrounding environment more comprehensively. On the horizontal axis, a higher value represents a greater computational power available within a smaller size and weight, indicating a stronger capability of the platform to support downstream tasks.
• Figure 4: The calibration pipeline of multi-fisheye camera set. The numbers indicate the calibration sequence. The letter on the top left of each box corresponds to the camera index in Fig. \ref{['fig:virtual_stereo']}.B
• Figure 5: Illustration of virtual-stereo frontend. A: the Z axis of virtual cameras (Orange and Blue) and the fisheye camera (Black). B: the placement of four fisheye cameras. C: shows the virtual stereo pairs.