Adaptive Sampling-based Particle Filter for Visual-inertial Gimbal in the Wild

TL;DR

The paper tackles robust, real-time gimbal stabilization for UAVs operating in natural terrains where feature-rich visual tracking is unreliable. It integrates a light-weight binary sky/ground segmentation ($\text{ResNet-18}$) with geometry cues from skyline and ground plane to estimate orientation, and employs a manifold-surface adaptive particle filter to fuse IMU and CV-driven orientations on a spherical domain. Key contributions include the edge-optimized CNN segmentation, skyline/ground-based rotation estimation, and a three-resolution adaptive sampling strategy that enhances robustness and redundancy in fusion on a Jetson Nano platform, demonstrated in outdoor scenarios. The approach yields improved accuracy and stability over single-sensor baselines and standard VO/VIO methods, enabling practical gimbal control in challenging outdoor conditions, while acknowledging skyline visibility as a limitation and suggesting hybrid extensions for broader scenes.

Abstract

In this paper, we present a Computer Vision (CV) based tracking and fusion algorithm, dedicated to a 3D printed gimbal system on drones operating in nature. The whole gimbal system can stabilize the camera orientation robustly in a challenging nature scenario by using skyline and ground plane as references. Our main contributions are the following: a) a light-weight Resnet-18 backbone network model was trained from scratch, and deployed onto the Jetson Nano platform to segment the image into binary parts (ground and sky); b) our geometry assumption from nature cues delivers the potential for robust visual tracking by using the skyline and ground plane as a reference; c) a spherical surface-based adaptive particle sampling, can fuse orientation from multiple sensor sources flexibly. The whole algorithm pipeline is tested on our customized gimbal module including Jetson and other hardware components. The experiments were performed on top of a building in the real landscape.

Figures (9)

• Figure 1: Demo of gimbal platform on the fixed-wing airplane.
• Figure 2: Open source hardware setup.
• Figure 3: Block diagram of the presented algorithm. Circular nodes in pink are signals or controlled targets. Rectangular boxes in yellow are ROS nodes for the algorithm, the dashed region is the front-end perception part, including tracking of skyline and ground plane.
• Figure 4: Failure case demo using OpenCV pipeline.
• Figure 5: Sample images and ground truth masks for training.