Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalizable Spacecraft Trajectory Generation via Multimodal Learning with Transformers

Davide Celestini, Amirhossein Afsharrad, Daniele Gammelli, Tommaso Guffanti, Gioele Zardini, Sanjay Lall, Elisa Capello, Simone D'Amico, Marco Pavone

TL;DR

This work presents a novel trajectory generation framework that generalizes across diverse problem configurations, by leveraging highcapacity transformer neural networks capable of learning from multimodal data sources, achieving up to 30% cost improvement and 80 % reduction in infeasible cases with respect to traditional approaches.

Abstract

Effective trajectory generation is essential for reliable on-board spacecraft autonomy. Among other approaches, learning-based warm-starting represents an appealing paradigm for solving the trajectory generation problem, effectively combining the benefits of optimization- and data-driven methods. Current approaches for learning-based trajectory generation often focus on fixed, single-scenario environments, where key scene characteristics, such as obstacle positions or final-time requirements, remain constant across problem instances. However, practical trajectory generation requires the scenario to be frequently reconfigured, making the single-scenario approach a potentially impractical solution. To address this challenge, we present a novel trajectory generation framework that generalizes across diverse problem configurations, by leveraging high-capacity transformer neural networks capable of learning from multimodal data sources. Specifically, our approach integrates transformer-based neural network models into the trajectory optimization process, encoding both scene-level information (e.g., obstacle locations, initial and goal states) and trajectory-level constraints (e.g., time bounds, fuel consumption targets) via multimodal representations. The transformer network then generates near-optimal initial guesses for non-convex optimization problems, significantly enhancing convergence speed and performance. The framework is validated through extensive simulations and real-world experiments on a free-flyer platform, achieving up to 30% cost improvement and 80% reduction in infeasible cases with respect to traditional approaches, and demonstrating robust generalization across diverse scenario variations.

Generalizable Spacecraft Trajectory Generation via Multimodal Learning with Transformers

TL;DR

This work presents a novel trajectory generation framework that generalizes across diverse problem configurations, by leveraging highcapacity transformer neural networks capable of learning from multimodal data sources, achieving up to 30% cost improvement and 80 % reduction in infeasible cases with respect to traditional approaches.

Abstract

Effective trajectory generation is essential for reliable on-board spacecraft autonomy. Among other approaches, learning-based warm-starting represents an appealing paradigm for solving the trajectory generation problem, effectively combining the benefits of optimization- and data-driven methods. Current approaches for learning-based trajectory generation often focus on fixed, single-scenario environments, where key scene characteristics, such as obstacle positions or final-time requirements, remain constant across problem instances. However, practical trajectory generation requires the scenario to be frequently reconfigured, making the single-scenario approach a potentially impractical solution. To address this challenge, we present a novel trajectory generation framework that generalizes across diverse problem configurations, by leveraging high-capacity transformer neural networks capable of learning from multimodal data sources. Specifically, our approach integrates transformer-based neural network models into the trajectory optimization process, encoding both scene-level information (e.g., obstacle locations, initial and goal states) and trajectory-level constraints (e.g., time bounds, fuel consumption targets) via multimodal representations. The transformer network then generates near-optimal initial guesses for non-convex optimization problems, significantly enhancing convergence speed and performance. The framework is validated through extensive simulations and real-world experiments on a free-flyer platform, achieving up to 30% cost improvement and 80% reduction in infeasible cases with respect to traditional approaches, and demonstrating robust generalization across diverse scenario variations.

Paper Structure

This paper contains 12 sections, 7 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Methodology
  4. Multimodal Trajectory Representation
  5. Transformer architecture for multimodal representation
  6. Pre-training
  7. Experiments
  8. Free-flyer dynamics and problem specification
  9. Experimental design
  10. Simulation experiments
  11. Experimental results
  12. Conclusions

Figures (6)

  • Figure 1: (a) Schematic representation of the free-flyer in the global reference frame $\mathrm{O}_{\mathrm{xy}}$. The eight-thruster configuration allows for independent control of rotational and translational motion. (b) A top-view of the free-flyer platform with highlighted start (black) and goal (green) regions.
  • Figure 2: Comparison between warm-starting trajectories generated by ART-Rs and REL in scenarios with varying obstacle configurations. The modification of the right-most obstacle prompts ART to adapt its trajectory accordingly, demonstrating coherent behavior in response to the change.
  • Figure 3: Percentage improvements in cost suboptimality (top) and number of SCP iterations (middle) relative to REL, along with a comparison of infeasibility rates (bottom), achieved by warm-starting the SCP using ART models trained on datasets with varying degrees of scenario diversity. Each bar represents the average improvement for constraint-violation-budgets (i.e., $\mathcal{C}_\mathrm{REL} (t_1)$) greater than or equal to the corresponding x-axis value.
  • Figure 4: Percentage improvements in cost suboptimality (top) and number of SCP iterations (middle) relative to REL, along with comparisons of infeasibility rates (bottom), achieved by warm-starting the SCP using ART models trained on datasets with varying degrees of final time diversity. Each bar represents the average improvement over all test samples corresponding to the final time $T_f$ indicated on the x-axis.
  • Figure 5: Trajectories planned using SCP ART (dashed lines) and tracked with a PID controller (solid lines) on the hardware testbed, varying obstacles' configuration (left) and final time (right). Videos: https://acc25art.github.io
  • ...and 1 more figures

Theorems & Definitions (1)

  • Example 1: The Autonomous Rendezvous Transformer