A Quality Diversity Approach to Evolving Model Rockets

TL;DR

This study applies Quality Diversity (QD) to evolve a diverse set of low-power rocket designs using MAP-Elites, CMA-ME, and CMA-MAE within an OpenRocket simulation framework. It shows CMA-ME achieves the broadest exploration of the design space and highest QD scores, challenging assumptions about CMA-MAE's superiority in all domains. Real-world launches reveal substantial sim-to-real gaps due to manufacturing variability and measurement limitations, though several evolved designs remain viable. The work demonstrates how QD can be an accessible, educational tool for iterative design and provides practical insights into algorithm performance for aerospace-inspired tasks.

Abstract

Model rocketry presents a design task accessible to undergraduates while remaining an interesting challenge. Allowing for variation in fins, nose cones, and body tubes presents a rich design space containing numerous ways to achieve various altitudes. Therefore, when exploring possible designs computationally, it makes sense to apply a method that produces various possibilities for decision-makers to choose from: Quality Diversity (QD). The QD methods MAP-Elites, CMA-ME, and CMA-MAE are applied to model rocket design using the open-source OpenRocket software to characterize the behavior and determine the fitness of evolved designs. Selected rockets were manufactured and launched to evaluate them in the real world. Simulation results demonstrate that CMA-ME produces the widest variety of rocket designs, which is surprising given that CMA-MAE is a more recent method designed to overcome shortcomings with CMA-ME. Real-world testing demonstrates that a wide range of standard and unconventional designs are viable, though issues with the jump from simulation to reality cause some rockets to perform unexpectedly. This paper provides a case study on applying QD to a task accessible to a broader audience than industrial engineering tasks and uncovers unexpected results about the relative performance of different QD algorithms.

Figures (7)

• Figure 1: Archive Cells Over Evolution: Average number of occupied archive cells at each generation. Each line is the average across 30 runs, depicted with 95% confidence intervals. CMA-ME is significantly better than CMA-MAE and MAP-Elites at filling the archive with designs, though all generate thousands of designs.
• Figure 2: QD Score Over Evolution: Average QD Score at each generation. Each line is the average across 30 runs, depicted with 95% confidence intervals. Other than scale, results are similar to Fig. \ref{['fig:occupied']}.
• Figure 3: Comparative Mega-Archive Coverage: All 30 experiments of each type had their archives combined into one mega-archive: MAP-Elites, CMA-ME, and CMA-MAE. Each cell indicates which algorithms cover it. At least one experiment out of 30 for each method fills most bins in the archive (brown). CMA-MAE fails to occupy some cells at lower altitudes for the ELLIPSOID, POWER, and PARABOLIC nose types (tan), and some of the lowest altitudes for CONICAL, ELLIPSOID, and POWER are only achieved by CMA-ME (green). At both the lowest and highest altitudes there are purple fringes indicating bins that only MAP-Elites could not reach. There are a few red bins at the highest altitudes that only CMA-MAE reached and some blue bins at the lowest altitudes that only MAP-Elites reached.
• Figure 4: Bin Occupancy Counts Across All 30 Archives of Each Algorithm: (\ref{['fig:map_elites_count']}) The intensity of each cell indicates the number of MAP-Elites runs that found a solution for that bin out of 30. MAP-Elites reliably reaches most mid-range altitudes but is less consistent for high and low altitudes, especially for designs with low stability at a low altitude. (\ref{['fig:cma_me_imp_count']}) CMA-ME is also consistent for mid-range altitudes and less consistent at the extremes. However, CMA-ME's coverage of the lowest altitudes is better than MAP-Elites. (\ref{['fig:cma_mae_count']}) CMA-MAE coverage is surprisingly weak in various areas, as evidenced by green streaks indicating that around 5 or more runs out of 30 did not discover any rockets in bins that were consistently reached by CMA-ME and MAP-Elites.
• Figure 5: Specific Archive Coverage and Performance: (\ref{['fig:map_elites0']}) Final archive from the first MAP-Elites run, from which specific rockets were selected for manufacture. Dark purple cells represent unstable designs. (\ref{['fig:cma_me_imp0']}) Final archive from the first CMA-ME run, from which specific rockets were selected for manufacture. It contains fewer unstable rockets than the MAP-Elites run and has better coverage. (\ref{['fig:cma_mae0']}) Final archive from the first CMA-MAE run, from which specific rockets were selected for manufacture. Its coverage is surprisingly worse than MAP-Elites.