Swarm-Gen: Fast Generation of Diverse Feasible Swarm Behaviors

TL;DR

This work tackles the challenge of generating diverse, feasible swarm trajectories between designated start and goal states. It blends two generative priors, CVAE and VQ-VAE, with a GPU-accelerated, differentiable Safety-Filter to enforce workspace and collision constraints, aided by a learned initialization network that speeds the fixed-point solver. The approach yields multiple, feasible multi-modal trajectories in real time on commodity GPUs, with CVAE offering faster inference and VQ-VAE providing greater diversity. By situating the method relative to trajectory prediction and diffusion-based approaches, the paper demonstrates a robust, constraint-satisfying alternative for real-time multi-robot coordination and data-driven swarm simulation.

Abstract

Coordination behavior in robot swarms is inherently multi-modal in nature. That is, there are numerous ways in which a swarm of robots can avoid inter-agent collisions and reach their respective goals. However, the problem of generating diverse and feasible swarm behaviors in a scalable manner remains largely unaddressed. In this paper, we fill this gap by combining generative models with a safety-filter (SF). Specifically, we sample diverse trajectories from a learned generative model which is subsequently projected onto the feasible set using the SF. We experiment with two choices for generative models, namely: Conditional Variational Autoencoder (CVAE) and Vector-Quantized Variational Autoencoder (VQ-VAE). We highlight the trade-offs these two models provide in terms of computation time and trajectory diversity. We develop a custom solver for our SF and equip it with a neural network that predicts context-specific initialization. Thecinitialization network is trained in a self-supervised manner, taking advantage of the differentiability of the SF solver. We provide two sets of empirical results. First, we demonstrate that we can generate a large set of multi-modal, feasible trajectories, simulating diverse swarm behaviors, within a few tens of milliseconds. Second, we show that our initialization network provides faster convergence of our SF solver vis-a-vis other alternative heuristics.

Figures (7)

• Figure 1: Architecture of our VQ-VAE pipeline. A CNN encoder compresses optimal trajectories (time-stamped waypoints) into a discrete latent space $\mathbf{Z}_q$. The decoder reconstructs polynomial coefficients $\overline{\xi}$ based on $\mathbf{Z}_q$, which is then converted to trajectories. The differentiable QP block ensures that the reconstructed trajectories satisfy the boundary conditions on start and goal states.
• Figure 2: Architecture of PixelCNN used to sample from the VQ-VAE. Its output is a multinomial distribution over $\mathbf{h}_q$ which can be used to generate different samples of it. These are then used to generate different $\mathbf{Z}_q$ and consequently different multi-robot trajectories.
• Figure 3: Training pipeline for learning context specific initialization for our SF solver. It consists of learnable layers (CNN+MLP) followed by an unrolled chain of our fixed-point solver underlying our SF. During training, the gradients of the loss function is traced through the fixed-point solver. This in turn ensures that the CNN and MLP layers are aware of how its predictions are leveraged by the downstream fixed-point solver.
• Figure 4: Our pipeline to fit CVAE to the dataset of expert multi-robot trajectories. The differentiable QP block is simply \ref{['eq::sf_form']} without the inequality constraints.
• Figure 5: Fig.(a) represents the fraction of the trajectories sampled from VQ-VAE/CVAE that becomes feasible after applying SF for 200 iterations. Fig.(b) compares the diversity of CVAE and VQ-VAE models through the cosine similarity metric computed over the feasible trajectories.