A Comparison of Deep Learning Architectures for Spacecraft Anomaly Detection

TL;DR

The paper addresses anomaly detection in spacecraft telemetry by systematically comparing 13 deep learning architectures across CNN, RNN/LSTM, Transformer, and hybrid models using the SMAP/MSL dataset. It introduces an unsupervised clustering approach to categorize telemetry channels by data-type and reports per-cluster architecture performance, achieving an ensemble-level average F1 anomaly of 84.7%, exceeding a baseline. The study finds CNN-based models like XceptionTimePlus perform well overall, while per-data-type improvements arise from tailoring models to specific telemetry patterns, suggesting ensemble, data-type-aware strategies. Practical considerations include CPU-only training and the lack of hyperparameter tuning, highlighting the potential for optimized ensembles and classification-based approaches in real-time spacecraft anomaly detection.

Abstract

Spacecraft operations are highly critical, demanding impeccable reliability and safety. Ensuring the optimal performance of a spacecraft requires the early detection and mitigation of anomalies, which could otherwise result in unit or mission failures. With the advent of deep learning, a surge of interest has been seen in leveraging these sophisticated algorithms for anomaly detection in space operations. This study aims to compare the efficacy of various deep learning architectures in detecting anomalies in spacecraft data. The deep learning models under investigation include Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) networks, and Transformer-based architectures. Each of these models was trained and validated using a comprehensive dataset sourced from multiple spacecraft missions, encompassing diverse operational scenarios and anomaly types. Initial results indicate that while CNNs excel in identifying spatial patterns and may be effective for some classes of spacecraft data, LSTMs and RNNs show a marked proficiency in capturing temporal anomalies seen in time-series spacecraft telemetry. The Transformer-based architectures, given their ability to focus on both local and global contexts, have showcased promising results, especially in scenarios where anomalies are subtle and span over longer durations. Additionally, considerations such as computational efficiency, ease of deployment, and real-time processing capabilities were evaluated. While CNNs and LSTMs demonstrated a balance between accuracy and computational demands, Transformer architectures, though highly accurate, require significant computational resources. In conclusion, the choice of deep learning architecture for spacecraft anomaly detection is highly contingent on the nature of the data, the type of anomalies, and operational constraints.

Figures (7)

• Figure 1: Anomaly Detection with Forecasting and Thresholding
• Figure 2: Clustering Process
• Figure 3: Elbow Method used to Determine Optimum Number of Clusters
• Figure 4: Resulting Data Clusters when k = 5
• Figure 5: Data Types per Cluster