Machine Learning in Space: Surveying the Robustness of on-board ML models to Radiation

TL;DR

The paper tackles the robustness of on-board ML in spacecraft to space radiation, a largely underexplored threat to ML-enabled pipelines. It combines a systematic literature review with a proof-of-concept study that uses manual weight perturbations and image disturbances on a U-Net for cloud detection, highlighting gaps in hazard-aware evaluation and reproducibility. It reveals that most prior work either ignores radiation/temperature effects or lacks accessible, open resources, and only a few propose genuine robustness strategies. To address this, the authors release their tools and data to enable replication and further research into space-tolerant ML, providing a practical framework for researchers to evaluate radiation-induced faults without field tests. They suggest future directions including training-time fault analysis and expanding beyond image data to broaden applicability.

Abstract

Modern spacecraft are increasingly relying on machine learning (ML). However, physical equipment in space is subject to various natural hazards, such as radiation, which may inhibit the correct operation of computing devices. Despite plenty of evidence showing the damage that naturally-induced faults can cause to ML-related hardware, we observe that the effects of radiation on ML models for space applications are not well-studied. This is a problem: without understanding how ML models are affected by these natural phenomena, it is uncertain "where to start from" to develop radiation-tolerant ML software. As ML researchers, we attempt to tackle this dilemma. By partnering up with space-industry practitioners specialized in ML, we perform a reflective analysis of the state of the art. We provide factual evidence that prior work did not thoroughly examine the impact of natural hazards on ML models meant for spacecraft. Then, through a "negative result", we show that some existing open-source technologies can hardly be used by researchers to study the effects of radiation for some applications of ML in satellites. As a constructive step forward, we perform simple experiments showcasing how to leverage current frameworks to assess the robustness of practical ML models for cloud detection against radiation-induced faults. Our evaluation reveals that not all faults are as devastating as claimed by some prior work. By publicly releasing our resources, we provide a foothold -- usable by researchers without access to spacecraft -- for spearheading development of space-tolerant ML models.

Figures (10)

• Figure 1: Applications of ML on-board spacecraft -- In some cases, ML is used to analyse data of Earth (taken by the satellite), and then send the results to Earth; in other cases, the output of ML is used by the satellite itself.
• Figure 2: Perspective of the ML researcher -- ML researchers do not have access to spacecraft or to physical equipment that reproduces a space setting. They only rely on open-source tools (models and data) and commodity hardware (e.g., GPUs), but their knowhow can help improve state-of-the-art methods for real-world deployments of ML.
• Figure 3: Using ML for on-board Cloud Detection -- The satellite acquires data (i.e., images) of Earth; such data may be subject to natural disturbances (e.g., radiation). Then, the captured data is analyzed by an ML model (which may also be subject natural disturbances). The output of this analysis is then sent back to Earth. To optimize downlink communications, "cloudy" images (detected via on-board ML) are not transferred. This saves bandwidth.
• Figure 4: Possible image disturbances -- The data (i.e., images) acquired by in-orbit satellites can be perturbed in many ways. Feeding such data to an ML model may "naturally" impact its performance. (Own figure, code is at: ourRepo)
• Figure 5: Our negative experiment -- We followed the guidelines provided by the developers of LLTFI. We could not finish the workflow due to a fatal error for which we found no workaround (even after consultation with practitioners).