Exploring Latent Pathways: Enhancing the Interpretability of Autonomous Driving with a Variational Autoencoder

TL;DR

The paper tackles the interpretability-performance trade-off in autonomous driving by substituting a CNN perception module with a Variational Autoencoder (VAE) and coupling it to a compact Neural Circuit Policy (NCP) controller, forming a VAE-NCP pipeline that directly generates steering commands. A key contribution is the Automatic Latent Perturbation (ALP) assistant, which automates latent-space perturbations to quantify and visualize the semantic information encoded in each latent dimension, linking latent changes to steering behavior. The approach jointly optimizes reconstruction, latent regularization, and steering prediction through a loss $L = \beta L_{recon} + \gamma L_{KL} + \alpha L_{pred}$, and leverages a sparse, temporally dynamic NCP with $19$ liquid time-constant neurons to maintain interpretability and efficiency. Empirical results on a $78\times200$ RGB-vision dataset show that VAE-NCP achieves favorable interpretability and competitive performance relative to CNN-based baselines, with tenfold cross-validation underscoring the trade-off between accuracy and transparency. The ALP tool enables targeted data collection and offers a pathway toward active learning and safer, more transparent autonomous driving systems.

Abstract

Autonomous driving presents a complex challenge, which is usually addressed with artificial intelligence models that are end-to-end or modular in nature. Within the landscape of modular approaches, a bio-inspired neural circuit policy model has emerged as an innovative control module, offering a compact and inherently interpretable system to infer a steering wheel command from abstract visual features. Here, we take a leap forward by integrating a variational autoencoder with the neural circuit policy controller, forming a solution that directly generates steering commands from input camera images. By substituting the traditional convolutional neural network approach to feature extraction with a variational autoencoder, we enhance the system's interpretability, enabling a more transparent and understandable decision-making process. In addition to the architectural shift toward a variational autoencoder, this study introduces the automatic latent perturbation tool, a novel contribution designed to probe and elucidate the latent features within the variational autoencoder. The automatic latent perturbation tool automates the interpretability process, offering granular insights into how specific latent variables influence the overall model's behavior. Through a series of numerical experiments, we demonstrate the interpretative power of the variational autoencoder-neural circuit policy model and the utility of the automatic latent perturbation tool in making the inner workings of autonomous driving systems more transparent.

Figures (5)

• Figure 1: A framework where a variational autoencoder and neural circuit policies work together, enhancing interpretability in autonomous driving from input to interpretable steering and visual outputs.
• Figure 2: VAE-NCP architecture: from image to steering command and reconstruction.
• Figure 3: ALP analyses across various images and latent dimensions. Each row showcases a different image-dimension combination, the differential image accentuates all the changes, the heatmap overlay on the original image points out the affected areas due to latent perturbations.
• Figure 4: Analysis of steering error variability across different perturbed dimensions on a sample of 1000 images. Top (a): the comparison of MSE at perturbations with two different standard deviations (-2$\sigma$ and +2$\sigma$) is shown for each dimension, highlighting the range of variability in the errors. Bottom (a): Standard deviation of the steering errors for each dimension. Plot (b): box plot of steering error for the whole dataset.
• Figure 5: Comparison of the impact score across different latent dimensions for high and low steering prediction error percentiles. The right plot illustrates the impact scores for the highest 10% of prediction errors, indicating dimensions with the most significant influence on steering errors. The left plot displays the impact scores for the lowest 10% of prediction errors.