Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An LSTM-based Test Selection Method for Self-Driving Cars

Ali Güllü, Faiz Ali Shah, Dietmar Pfahl

TL;DR

The paper addresses the high cost of simulation-based testing for self-driving cars by proposing ITS4SDC, an LSTM-based test selector that treats road coordinates as sequences to classify tests as SAFE or UNSAFE. The model is trained and evaluated on Frenetic-generated roads in the BeamNG.tech environment, using a bidirectional LSTM with 220 cells and compared against the SDC-Scissor baseline across two datasets. ITS4SDC demonstrates higher accuracy and precision, with competitive recall and F1, particularly when trained on larger datasets, indicating strong potential to reduce testing overhead while maintaining safety validation. The work highlights the value of sequence-based road data and opens avenues for future improvements through larger datasets and attention-based architectures.

Abstract

Self-driving cars require extensive testing, which can be costly in terms of time. To optimize this process, simple and straightforward tests should be excluded, focusing on challenging tests instead. This study addresses the test selection problem for lane-keeping systems for self-driving cars. Road segment features, such as angles and lengths, were extracted and treated as sequences, enabling classification of the test cases as "safe" or "unsafe" using a long short-term memory (LSTM) model. The proposed model is compared against machine learning-based test selectors. Results demonstrated that the LSTM-based method outperformed machine learning-based methods in accuracy and precision metrics while exhibiting comparable performance in recall and F1 scores. This work introduces a novel deep learning-based approach to the road classification problem, providing an effective solution for self-driving car test selection using a simulation environment.

An LSTM-based Test Selection Method for Self-Driving Cars

TL;DR

The paper addresses the high cost of simulation-based testing for self-driving cars by proposing ITS4SDC, an LSTM-based test selector that treats road coordinates as sequences to classify tests as SAFE or UNSAFE. The model is trained and evaluated on Frenetic-generated roads in the BeamNG.tech environment, using a bidirectional LSTM with 220 cells and compared against the SDC-Scissor baseline across two datasets. ITS4SDC demonstrates higher accuracy and precision, with competitive recall and F1, particularly when trained on larger datasets, indicating strong potential to reduce testing overhead while maintaining safety validation. The work highlights the value of sequence-based road data and opens avenues for future improvements through larger datasets and attention-based architectures.

Abstract

Self-driving cars require extensive testing, which can be costly in terms of time. To optimize this process, simple and straightforward tests should be excluded, focusing on challenging tests instead. This study addresses the test selection problem for lane-keeping systems for self-driving cars. Road segment features, such as angles and lengths, were extracted and treated as sequences, enabling classification of the test cases as "safe" or "unsafe" using a long short-term memory (LSTM) model. The proposed model is compared against machine learning-based test selectors. Results demonstrated that the LSTM-based method outperformed machine learning-based methods in accuracy and precision metrics while exhibiting comparable performance in recall and F1 scores. This work introduces a novel deep learning-based approach to the road classification problem, providing an effective solution for self-driving car test selection using a simulation environment.
Paper Structure (18 sections, 1 equation, 6 figures, 5 tables)

This paper contains 18 sections, 1 equation, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. SDC Scenarios
  4. SDC-Scissor Classification Approach
  5. BeamNG.tech Simulation Environment
  6. Road Generation for SDC Testing
  7. Related Work
  8. Development of the ITS4SDC Model
  9. Configuration of the Simulation Environment
  10. Dataset Preparation
  11. Building the ITS4SDC model Using LSTM
  12. Identifying road features
  13. Implementation of the ITS4SDC model
  14. Evaluation of the ITS4SDC Model
  15. Results
  16. ...and 3 more sections

Figures (6)

  • Figure 1: The vehicle stays in the lane (PASS).
  • Figure 2: The vehicle goes off the lane while taking a turn (FAIL).
  • Figure 3: The road coordinates of a test case.
  • Figure 4: Angle and length pairs corresponding to the road coordinates
  • Figure 5: Calculated angle information relative to previous reference segments.
  • ...and 1 more figures