Predicting UAV Type: An Exploration of Sampling and Data Augmentation for Time Series Classification

TL;DR

A machine learning model for classifying unmanned aerial vehicles as quadrotor, hexarotor, or fixed-wing is presented, which effectively applies a Long-Short Term Memory (LSTM) neural network for the purpose of time series classification.

Abstract

Unmanned aerial vehicles are becoming common and have many productive uses. However, their increased prevalence raises safety concerns -- how can we protect restricted airspace? Knowing the type of unmanned aerial vehicle can go a long way in determining any potential risks it carries. For instance, fixed-wing craft can carry more weight over longer distances, thus potentially posing a more significant threat. This paper presents a machine learning model for classifying unmanned aerial vehicles as quadrotor, hexarotor, or fixed-wing. Our approach effectively applies a Long-Short Term Memory (LSTM) neural network for the purpose of time series classification. We performed experiments to test the effects of changing the timestamp sampling method and addressing the imbalance in the class distribution. Through these experiments, we identified the top-performing sampling and class imbalance fixing methods. Averaging the macro f-scores across 10 folds of data, we found that the majority quadrotor class was predicted well (98.16%), and, despite an extreme class imbalance, the model could also predicted a majority of fixed-wing flights correctly (73.15%). Hexarotor instances were often misclassified as quadrotors due to the similarity of multirotors in general (42.15%). However, results remained relatively stable across certain methods, which prompted us to analyze and report on their tradeoffs. The supplemental material for this paper, including the code and data for running all the experiments and generating the results tables, is available at https://osf.io/mnsgk/.

Figures (3)

• Figure 1: Example of two flights for each vehicle type, where the top row represents manual and semi-guided flights and the bottom row represents mostly automated flights. The color represents different flight modes: Auto, Manual, Stabilized, Position Control, and Altitude Control. Auto encompasses taking off or landing as well as any waypoint-based travel. The other color represents different manual control methods, from pure manual (red) to some assistance from the UAV (the other colors). Fixed-wing vehicles have the same characteristics when flying a mission or manually being flown. On the other hand, multirotor vehicles' flight characteristic change depending if their auto, straight direct lines, or manual, more curved flight paths.
• Figure 2: A stacked bar graph showing the quantity of each UAV type for flights up to 1000 seconds. The x-axis is the duration of the flight binned in 10-second increments. Quadrotors make up most of the UAV types. Almost all flights are less than 10 minutes long with a majority under 30 seconds since PX4 review is often used to test UAVs.
• Figure 3: The two figures illustrate how fixed window average and average sampling work on flights with different flight times. For average sampling, the amount of data averaged changes across flights, with shorter flights averaging a smaller portion of the data. Fixed window average averages the same amount of data across flights but varies the gaps. For each flight, the algorithm uses the longest feature for sampling. For any shorter feature, we sample partially or not at all to guarantee that the sampling is aligned within a flight.