Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scaling Planning for Automated Driving using Simplistic Synthetic Data

Martin Stoll, Markus Mazzola, Maxim Dolgov, Jürgen Mathes, Nicolas Möser

TL;DR

The paper tackles scalable automated-driving planning by showing that targeted, simplistic synthetic data can replace large real-world datasets. It introduces a lightweight planning model operating on a binary BEV grid and trained with vanilla behavioural cloning, augmented by an iterative sim-to-real cycle that adds challenging scenarios through a simple simulator. Key findings demonstrate high real-world driving performance after targeted data augmentation, notable safety gains from including critical simulated scenarios, and strong limitations of relying solely on large public datasets. The results suggest a practical, modular pathway to scale planning without prohibitive data or realism requirements, with iterative testing and augmentation driving continual improvement.

Abstract

We challenge the perceived consensus that the application of deep learning to solve the automated driving planning task necessarily requires huge amounts of real-world data or highly realistic simulation. Focusing on a roundabout scenario, we show that this requirement can be relaxed in favour of targeted, simplistic simulated data. A benefit is that such data can be easily generated for critical scenarios that are typically underrepresented in realistic datasets. By applying vanilla behavioural cloning almost exclusively to lightweight simulated data, we achieve reliable and comfortable driving in a real-world test vehicle. We leverage an incremental development approach that includes regular in-vehicle testing to identify sim-to-real gaps, targeted data augmentation, and training scenario variations. In addition to a detailed description of the methodology, we share our lessons learned, touching upon scenario generation, simulation features, and evaluation metrics.

Scaling Planning for Automated Driving using Simplistic Synthetic Data

TL;DR

The paper tackles scalable automated-driving planning by showing that targeted, simplistic synthetic data can replace large real-world datasets. It introduces a lightweight planning model operating on a binary BEV grid and trained with vanilla behavioural cloning, augmented by an iterative sim-to-real cycle that adds challenging scenarios through a simple simulator. Key findings demonstrate high real-world driving performance after targeted data augmentation, notable safety gains from including critical simulated scenarios, and strong limitations of relying solely on large public datasets. The results suggest a practical, modular pathway to scale planning without prohibitive data or realism requirements, with iterative testing and augmentation driving continual improvement.

Abstract

We challenge the perceived consensus that the application of deep learning to solve the automated driving planning task necessarily requires huge amounts of real-world data or highly realistic simulation. Focusing on a roundabout scenario, we show that this requirement can be relaxed in favour of targeted, simplistic simulated data. A benefit is that such data can be easily generated for critical scenarios that are typically underrepresented in realistic datasets. By applying vanilla behavioural cloning almost exclusively to lightweight simulated data, we achieve reliable and comfortable driving in a real-world test vehicle. We leverage an incremental development approach that includes regular in-vehicle testing to identify sim-to-real gaps, targeted data augmentation, and training scenario variations. In addition to a detailed description of the methodology, we share our lessons learned, touching upon scenario generation, simulation features, and evaluation metrics.
Paper Structure (25 sections, 2 equations, 3 figures, 7 tables)

This paper contains 25 sections, 2 equations, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Model Description
  3. Input Representation
  4. Output Representation
  5. Network Architecture
  6. Trajectory Controller
  7. Training Setup
  8. Data Generation
  9. Augmentations
  10. Computer-Vision-Based Augmentations
  11. Off-Road Vehicles
  12. Traffic Vehicle Size
  13. Bitflip Augmentation
  14. Model Training
  15. Evaluation
  16. ...and 10 more sections

Figures (3)

  • Figure 1: Alternating steps of our itertive development cycle: train and optimise planner with simplistic simulation (left), observe sim2real gap in on-road test and extend simulator (right).
  • Figure 2: Model architecture consisting of a CNN backbone, a waypoints head, and a prediction head that generates the auxiliary prediction output.
  • Figure 3: Test vehicle equipped with a prototypic autonomous driving stack.