LASIL: Learner-Aware Supervised Imitation Learning For Long-term Microscopic Traffic Simulation

TL;DR

This paper tackles the challenge of realistic, long-term microscopic traffic simulation by addressing covariate shift in multi-agent imitation learning. It introduces LASIL, a learner-aware supervised imitation learning framework that leverages a context-conditioned variational autoencoder to augment expert states, aligning them with the learner’s distribution without requiring expert policy access. LASIL combines an edge-enhanced graph attention network with a policy trained via negative log-likelihood, plus post-processing steps (on-road projection and LQR smoothing) to produce stable, realistic trajectories. Evaluations on the real-world pNEUMA dataset show LASIL outperforms state-of-the-art baselines in short-term microscopic realism and long-term macroscopic realism, enabling simulations over 10 minutes with substantial improvements (up to 40x longer than prior methods) and providing a practical tool for urban traffic analysis and planning.

Abstract

Microscopic traffic simulation plays a crucial role in transportation engineering by providing insights into individual vehicle behavior and overall traffic flow. However, creating a realistic simulator that accurately replicates human driving behaviors in various traffic conditions presents significant challenges. Traditional simulators relying on heuristic models often fail to deliver accurate simulations due to the complexity of real-world traffic environments. Due to the covariate shift issue, existing imitation learning-based simulators often fail to generate stable long-term simulations. In this paper, we propose a novel approach called learner-aware supervised imitation learning to address the covariate shift problem in multi-agent imitation learning. By leveraging a variational autoencoder simultaneously modeling the expert and learner state distribution, our approach augments expert states such that the augmented state is aware of learner state distribution. Our method, applied to urban traffic simulation, demonstrates significant improvements over existing state-of-the-art baselines in both short-term microscopic and long-term macroscopic realism when evaluated on the real-world dataset pNEUMA.

Figures (6)

• Figure 1: Overview of our approach. The processes with purple and red arrows handle only expert or learner data, respectively, while the processes represented by black arrows apply to both data. Each state is a multi-agent state represented by a graph. Each model, including the VAE encoder, decoder, and policy network, is implemented using an EGAT network.
• Figure 2: Mean density and speed on each road over all time steps in the long-term evaluation. Our method's density and speed hot-maps have a more similar color to the ground-truth one compared with the SUMO's.
• Figure 3: Runtime of each time step during the long-term evaluation.
• Figure 4: Trajectories augmented by our context-conditioned VAE, predicted by our policy network and subsequent planned trajectory by the LQR module in lane keeping, turning and lane changing scenarios.
• Figure 5: Distributions of speed and leader vehicle's distance in long-term evaluation.