Learning Sampling Distribution and Safety Filter for Autonomous Driving with VQ-VAE and Differentiable Optimization

TL;DR

The paper tackles trajectory sampling for autonomous driving by replacing CVAE's Gaussian latent prior with a discrete latent space via Vector-Quantized VAE (VQ-VAE), learned from multimodal demonstrations to capture multi-modality in driving behaviors. A PixelCNN conditioned on observations samples codes from the discrete latent space, which the VQ-VAE decoder then translates into velocity and lateral-offset setpoints that are refined by a differentiable QP-based trajectory optimizer. A learnable safety filter based on barrier functions minimally corrects samples to satisfy collision avoidance and lane-boundary constraints, with parameters learned through self-supervised backpropagation through the optimization layers. Empirical results show up to a 12x reduction in collision-rate compared to a CVAE baseline in dense traffic, while maintaining competitive speeds and robustness under reduced compute budgets.

Abstract

Sampling trajectories from a distribution followed by ranking them based on a specified cost function is a common approach in autonomous driving. Typically, the sampling distribution is hand-crafted (e.g a Gaussian, or a grid). Recently, there have been efforts towards learning the sampling distribution through generative models such as Conditional Variational Autoencoder (CVAE). However, these approaches fail to capture the multi-modality of the driving behaviour due to the Gaussian latent prior of the CVAE. Thus, in this paper, we re-imagine the distribution learning through vector quantized variational autoencoder (VQ-VAE), whose discrete latent-space is well equipped to capture multi-modal sampling distribution. The VQ-VAE is trained with demonstration data of optimal trajectories. We further propose a differentiable optimization based safety filter to minimally correct the VQVAE sampled trajectories to ensure collision avoidance. We use backpropagation through the optimization layers in a self-supervised learning set-up to learn good initialization and optimal parameters of the safety filter. We perform extensive comparisons with state-of-the-art CVAE-based baseline in dense and aggressive traffic scenarios and show a reduction of up to 12 times in collision-rate while being competitive in driving speeds.

Figures (9)

• Figure 1: Comparison between trajectory distribution sampled from CVAE (a) and VQ-VAE (b). As can be seen, VQ-VAE shows higher diversity and multi-modality in the sampled trajectories. This can be further validated by Fig. (c) and (d) which show the Kernel Density Estimation plots of the forward velocity ($v_d$) and lateral-offset ($y_d$) associated with each sampled trajectory.
• Figure 2: Our overall pipeline that consists of sampling from a learnd VQ-VAE and passing the samples to a learned safety filter. This is followed by cost evaluation on the trajectory samples and the selection of the best trajectory.
• Figure 3: VQVAE with a differentiable QP block for learning discrete latent prior over the forward velocity $v_d$, lateral offset $y_d$ and the associated trajectory.
• Figure 4: The auto-regressive conditional PixelCNN architecture that is used to learn how to sample from the discrete latent space of a trained VQ-VAE.
• Figure 5: Learned Safety filter that consists of an MLP augmented with an optimization layer. We train the safety filter in self-supervised setting to learn the parameters of inequality constraints along with good intialization for the optimization problem.

Theorems & Definitions (1)

• Remark 1