Morphology and Behavior Co-Optimization of Modular Satellites for Attitude Control

TL;DR

This work addresses the challenge of attitude control for modular satellites by jointly optimizing morphology and control within a unified reinforcement-learning framework. It introduces a gradient-based co-design approach that uses a shared neural network with Neural Cellular Automata for morphology and a TD3-based policy for control, all embedded in a single MDP. Monte Carlo simulations show that the RL-based co-design achieves faster learning and higher final performance than evolution-based methods, producing morphologies with symmetric, balanced structures that bolster attitude performance. The study demonstrates the feasibility and benefits of brain-body co-design for modular spacecraft and highlights avenues for scaling to larger design spaces and real-world deployment.

Abstract

The emergence of modular satellites marks a significant transformation in spacecraft engineering, introducing a new paradigm of flexibility, resilience, and scalability in space exploration endeavors. In addressing complex challenges such as attitude control, both the satellite's morphological architecture and the controller are crucial for optimizing performance. Despite substantial research on optimal control, there remains a significant gap in developing optimized and practical assembly strategies for modular satellites tailored to specific mission constraints. This research gap primarily arises from the inherently complex nature of co-optimizing design and control, a process known for its notorious bi-level optimization loop. Conventionally tackled through artificial evolution, this issue involves optimizing the morphology based on the fitness of individual controllers, which is sample-inefficient and computationally expensive. In this paper, we introduce a novel gradient-based approach to simultaneously optimize both morphology and control for modular satellites, enhancing their performance and efficiency in attitude control missions. Our Monte Carlo simulations demonstrate that this co-optimization approach results in modular satellites with better mission performance compared to those designed by evolution-based approaches. Furthermore, this study discusses potential avenues for future research.

Figures (9)

• Figure 1: The concept of iBOSS modular satellites kreisel2019gamezhang2023modularity, which contains intelligent building blocks for on-orbit satellite servicing.
• Figure 2: TD3 improves stability in continuous action spaces by using two critic networks, an actor network, and their corresponding target networks. During training, the agent collects experiences by interacting with the environment, storing state transitions in a replay buffer. The training procedure repeats iteratively to refine the policy and improve performance.
• Figure 3: Schematic diagram of coordinate systems used in this work.
• Figure 4: Representation of the modular satellite.
• Figure 5: Modular satellite centroid translation reference coordinate system.