Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GOOSE: Goal-Conditioned Reinforcement Learning for Safety-Critical Scenario Generation

Joshua Ransiek, Johannes Plaum, Jacob Langner, Eric Sax

TL;DR

GOOSE tackles the challenge of generating safety-critical driving scenarios for ADAS/ADS testing by casting scenario generation as a goal-conditioned reinforcement learning problem. It leverages NURBS to model and efficiently manipulate adversarial trajectories in the Frenet frame, guided by constraint-based goals inspired by the OpenSCENARIO DSL. The approach trains a goal-conditioned policy using DroQ with HER, achieving data-efficient learning and outperforming baselines on three UN Regulation No. 157 ALKS-based scenarios. Experimental results demonstrate that GOOSE can reliably generate controllable, realistic critical events, offering a practical, standards-aligned tool for scenario-based safety testing. The work also suggests future paths toward multi-agent extensions and applying the methodology to real-world data to further enhance ADS robustness.

Abstract

Scenario-based testing is considered state-of-the-art for verifying and validating Advanced Driver Assistance Systems (ADASs) and Automated Driving Systems (ADSs). However, the practical application of scenario-based testing requires an efficient method to generate or collect the scenarios that are needed for the safety assessment. In this paper, we propose Goal-conditioned Scenario Generation (GOOSE), a goal-conditioned reinforcement learning (RL) approach that automatically generates safety-critical scenarios to challenge ADASs or ADSs. In order to simultaneously set up and optimize scenarios, we propose to control vehicle trajectories at the scenario level. Each step in the RL framework corresponds to a scenario simulation. We use Non-Uniform Rational B-Splines (NURBS) for trajectory modeling. To guide the goal-conditioned agent, we formulate test-specific, constraint-based goals inspired by the OpenScenario Domain Specific Language(DSL). Through experiments conducted on multiple pre-crash scenarios derived from UN Regulation No. 157 for Active Lane Keeping Systems (ALKS), we demonstrate the effectiveness of GOOSE in generating scenarios that lead to safety-critical events.

GOOSE: Goal-Conditioned Reinforcement Learning for Safety-Critical Scenario Generation

TL;DR

GOOSE tackles the challenge of generating safety-critical driving scenarios for ADAS/ADS testing by casting scenario generation as a goal-conditioned reinforcement learning problem. It leverages NURBS to model and efficiently manipulate adversarial trajectories in the Frenet frame, guided by constraint-based goals inspired by the OpenSCENARIO DSL. The approach trains a goal-conditioned policy using DroQ with HER, achieving data-efficient learning and outperforming baselines on three UN Regulation No. 157 ALKS-based scenarios. Experimental results demonstrate that GOOSE can reliably generate controllable, realistic critical events, offering a practical, standards-aligned tool for scenario-based safety testing. The work also suggests future paths toward multi-agent extensions and applying the methodology to real-world data to further enhance ADS robustness.

Abstract

Scenario-based testing is considered state-of-the-art for verifying and validating Advanced Driver Assistance Systems (ADASs) and Automated Driving Systems (ADSs). However, the practical application of scenario-based testing requires an efficient method to generate or collect the scenarios that are needed for the safety assessment. In this paper, we propose Goal-conditioned Scenario Generation (GOOSE), a goal-conditioned reinforcement learning (RL) approach that automatically generates safety-critical scenarios to challenge ADASs or ADSs. In order to simultaneously set up and optimize scenarios, we propose to control vehicle trajectories at the scenario level. Each step in the RL framework corresponds to a scenario simulation. We use Non-Uniform Rational B-Splines (NURBS) for trajectory modeling. To guide the goal-conditioned agent, we formulate test-specific, constraint-based goals inspired by the OpenScenario Domain Specific Language(DSL). Through experiments conducted on multiple pre-crash scenarios derived from UN Regulation No. 157 for Active Lane Keeping Systems (ALKS), we demonstrate the effectiveness of GOOSE in generating scenarios that lead to safety-critical events.
Paper Structure (21 sections, 12 equations, 7 figures, 2 tables, 1 algorithm)

This paper contains 21 sections, 12 equations, 7 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reinforcement Learning for Software Testing
  4. Goal-conditioned Reinforcement Learning
  5. Safety-Critical Traffic Scenario Generation
  6. Background
  7. Goal-conditioned Markov Decision Process
  8. Non-Uniform Rational B-Splines
  9. Method
  10. Action Space Definition
  11. State Space Definition
  12. Constraint-based Goal Definition
  13. Algorithm Summary
  14. Experiments and Results
  15. Implementation Details
  16. ...and 6 more sections

Figures (7)

  • Figure 1: The Pipeline for safety-critical scenario generation using a goal-conditioned RL agent. The agent (blue) attempts to increase the criticality of the simulated scenario by modifying the adversarial vehicle's trajectory, while the ego vehicle (yellow) is trying to maintain a safe state.
  • Figure 2: Exemplary NURBS curve of 3-th degree with weights $\omega_{0,...,3} = 1$ and the associated basis functions $N_{\{0,...,3\},3}$
  • Figure 3: Method for scenario generation using a goal-conditioned RL agent. The agent (red) attempts to increase the criticality of the simulated scenario by modifying the target vehicle's trajectory based on state and goal, while the ego vehicle (yellow) is trying to maintain a safe state.
  • Figure 4: Scheme outlining the application of an action to a vehicle trajectory
  • Figure 5: Initial states for the deceleration, cut-in and cut-out scenarios
  • ...and 2 more figures