Test Schedule Generation for Acceptance Testing of Mission-Critical Satellite Systems

TL;DR

The paper tackles the complex problem of scheduling In-Orbit Testing (IOT) campaigns for mission-critical satellites under constraints like antenna reconfiguration, satellite visibility, and resource costs. It introduces a multi-objective NSGA-III optimization framework that models passes, procedures, and slot allocations, and uses a conflict-graph representation to enforce feasibility. Industrial evaluation on Galileo GNSS data shows substantial gains over random search, ACO, and manual schedules, including large improvements in $fitcost$ and $fituse$, and the ability to present multiple viable Pareto-optimal schedules for trade-off analysis. Practitioner feedback highlights automation benefits, with future work focusing on diversity, risk integration, and broader constellation applicability.

Abstract

Mission-critical system, such as satellite systems, healthcare systems, and nuclear power plant control systems, undergo rigorous testing to ensure they meet specific operational requirements throughout their operation. This includes Operational Acceptance Testing (OAT), which aims to ensure that the system functions correctly under real-world operational conditions. In satellite development, In-Orbit Testing (IOT) is a crucial OAT activity performed regularly and as needed after deployment in orbit to check the satellite's performance and ensure that operational requirements are met. The scheduling of an IOT campaign, which executes multiple IOT procedures, is an important yet challenging problem, as it accounts for various factors, including satellite visibility, antenna usage costs, testing time periods, and operational constraints. To address the IOT scheduling problem, we propose a multi-objective approach to generate near-optimal IOT schedules, accounting for operational costs, fragmentation (i.e., the splitting of tests), and resource efficiency, which align with practitioners' objectives for IOT scheduling. Our industrial case study with SES Techcom shows significant improvements, as follows: an average improvement of 49.4% in the cost objective, 60.4% in the fragmentation objective, and 30% in the resource usage objective, compared to our baselines. Additionally, our approach improves cost efficiency by 538% and resource usage efficiency by 39.42% compared to manually constructed schedules provided by practitioners, while requiring only 12.5% of the time needed for manual IOT scheduling.

Figures (7)

• Figure 1: Example of a slot schedule and the corresponding slots.
• Figure 2: Three conflicting test procedures.
• Figure 3: Example of a conflict graph created from the passes of three satellites (A, B, and C)
• Figure 4: Comparing GSC and RS in terms of fitcost, fitfrag, and fituse. For brevity, we present the fitness values: the higher the fitness value, the better.
• Figure 5: Comparing the progression of the constraint violations between GSC and RS.