Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Is That Rain? Understanding Effects on Visual Odometry Performance for Autonomous UAVs and Efficient DNN-based Rain Classification at the Edge

Andrea Albanese, Yanran Wang, Davide Brunelli, David Boyle

TL;DR

This work probes how rainy conditions degrade camera-based visual odometry for autonomous UAVs and introduces a large open dataset (~335k images) spanning seven rain classes with direct lens exposure. It demonstrates that worst-case VO tracking error can be substantial under rain and benchmarks three edge-friendly DNNs for real-time rain condition classification, with MobileNetV3 Small achieving ~90% accuracy and ~93 FPS on constrained hardware. By enabling real-time disturbance estimation, the approach can feed into flight controllers to select countermeasures and improve reliability, while the dataset and models also open avenues for finer-grained local weather inference. Overall, the study provides both quantitative insights into rain-induced VO degradation and a practical, low-footprint solution for rain awareness in autonomous UAV navigation.

Abstract

The development of safe and reliable autonomous unmanned aerial vehicles relies on the ability of the system to recognise and adapt to changes in the local environment based on sensor inputs. State-of-the-art local tracking and trajectory planning are typically performed using camera sensor input to the flight control algorithm, but the extent to which environmental disturbances like rain affect the performance of these systems is largely unknown. In this paper, we first describe the development of an open dataset comprising ~335k images to examine these effects for seven different classes of precipitation conditions and show that a worst-case average tracking error of 1.5 m is possible for a state-of-the-art visual odometry system (VINS-Fusion). We then use the dataset to train a set of deep neural network models suited to mobile and constrained deployment scenarios to determine the extent to which it may be possible to efficiently and accurately classify these `rainy' conditions. The most lightweight of these models (MobileNetV3 small) can achieve an accuracy of 90% with a memory footprint of just 1.28 MB and a frame rate of 93 FPS, which is suitable for deployment in resource-constrained and latency-sensitive systems. We demonstrate a classification latency in the order of milliseconds using typical flight computer hardware. Accordingly, such a model can feed into the disturbance estimation component of an autonomous flight controller. In addition, data from unmanned aerial vehicles with the ability to accurately determine environmental conditions in real time may contribute to developing more granular timely localised weather forecasting.

Is That Rain? Understanding Effects on Visual Odometry Performance for Autonomous UAVs and Efficient DNN-based Rain Classification at the Edge

TL;DR

This work probes how rainy conditions degrade camera-based visual odometry for autonomous UAVs and introduces a large open dataset (~335k images) spanning seven rain classes with direct lens exposure. It demonstrates that worst-case VO tracking error can be substantial under rain and benchmarks three edge-friendly DNNs for real-time rain condition classification, with MobileNetV3 Small achieving ~90% accuracy and ~93 FPS on constrained hardware. By enabling real-time disturbance estimation, the approach can feed into flight controllers to select countermeasures and improve reliability, while the dataset and models also open avenues for finer-grained local weather inference. Overall, the study provides both quantitative insights into rain-induced VO degradation and a practical, low-footprint solution for rain awareness in autonomous UAV navigation.

Abstract

The development of safe and reliable autonomous unmanned aerial vehicles relies on the ability of the system to recognise and adapt to changes in the local environment based on sensor inputs. State-of-the-art local tracking and trajectory planning are typically performed using camera sensor input to the flight control algorithm, but the extent to which environmental disturbances like rain affect the performance of these systems is largely unknown. In this paper, we first describe the development of an open dataset comprising ~335k images to examine these effects for seven different classes of precipitation conditions and show that a worst-case average tracking error of 1.5 m is possible for a state-of-the-art visual odometry system (VINS-Fusion). We then use the dataset to train a set of deep neural network models suited to mobile and constrained deployment scenarios to determine the extent to which it may be possible to efficiently and accurately classify these `rainy' conditions. The most lightweight of these models (MobileNetV3 small) can achieve an accuracy of 90% with a memory footprint of just 1.28 MB and a frame rate of 93 FPS, which is suitable for deployment in resource-constrained and latency-sensitive systems. We demonstrate a classification latency in the order of milliseconds using typical flight computer hardware. Accordingly, such a model can feed into the disturbance estimation component of an autonomous flight controller. In addition, data from unmanned aerial vehicles with the ability to accurately determine environmental conditions in real time may contribute to developing more granular timely localised weather forecasting.
Paper Structure (16 sections, 4 figures, 9 tables)

This paper contains 16 sections, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Experimental Setup
  4. Hardware Specification
  5. Mechanical Design for Moisture Protection
  6. Visual Odometry Algorithm
  7. Experimental Environment and Settings
  8. Dataset and DNN Development
  9. Dataset
  10. Deep Neural Network Specifications
  11. Experimental Results & Analysis
  12. VO System in Static and Moving Conditions
  13. Static
  14. Moving
  15. DNN Test
  16. ...and 1 more sections

Figures (4)

  • Figure 1: The water-resistant box that hosts the processing unit and the depth camera. Right hand side shows stool mounting for moving conditions (Sec.V).
  • Figure 2: Example of dataset images for each class of rain.
  • Figure 3: Trajectory estimation and data distribution of the static scenario under the different rain conditions.
  • Figure 4: Trajectory estimation and data distribution of the moving scenario under the different rain conditions.