Deep Reinforcement Learning for Adverse Garage Scenario Generation

TL;DR

The paper tackles the challenge of efficiently generating diverse, adverse underground garage scenarios for autonomous valet parking testing by reframing static scene construction as a deep reinforcement learning problem. It introduces a procedural content generation framework that learns to produce encoding matrices representing garage layouts, which are then converted into FBX and OpenDrive files to populate Carla environments. Key contributions include a mathematical block taxonomy and a state-machine coloring scheme, a reward structure that balances constraint satisfaction with usability goals, and metrics for coverage and difficulty validated through RL experiments and AVP cruise tests. The approach enables scalable, customizable, and realistic garage scenarios that improve test coverage and the assessment of planning algorithms in autonomous driving systems.

Abstract

Autonomous vehicles need to travel over 11 billion miles to ensure their safety. Therefore, the importance of simulation testing before real-world testing is self-evident. In recent years, the release of 3D simulators for autonomous driving, represented by Carla and CarSim, marks the transition of autonomous driving simulation testing environments from simple 2D overhead views to complex 3D models. During simulation testing, experimenters need to build static scenes and dynamic traffic flows, pedestrian flows, and other experimental elements to construct experimental scenarios. When building static scenes in 3D simulators, experimenters often need to manually construct 3D models, set parameters and attributes, which is time-consuming and labor-intensive. This thesis proposes an automated program generation framework. Based on deep reinforcement learning, this framework can generate different 2D ground script codes, on which 3D model files and map model files are built. The generated 3D ground scenes are displayed in the Carla simulator, where experimenters can use this scene for navigation algorithm simulation testing.

Figures (17)

• Figure 1: The architecture of an autonomous driving system.
• Figure 2: A example of automated valet parking. Red lines represent the predicted path of the car deployed with ADS.
• Figure 3: The framework of MDPs.
• Figure 4: Three combinations of vertical parking space.
• Figure 5: The PCG Framework. It starts at the RL framework. RL framework needs the input of one initial map and parameters. The train process of RL framework generates the encoding matrix set and they can be transformed into the road network matrix set. The road network matrix set will generate fbx files and xodr files, which can import into Carla simulator as the 3D map.