Adapting Reinforcement Learning for Path Planning in Constrained Parking Scenarios

TL;DR

The paper tackles real-time path planning in highly constrained parking under imperfect perception. It presents a Deep Reinforcement Learning planner grounded in a bicycle-model dynamic, with an ego-centric input representation and a cross-attention module, augmented by an action-chunking wrapper and curriculum learning, operating on state $(x, y, \theta, \delta)$. To enable rigorous evaluation, it introduces ParkBench, an open-source benchmark with 51 rear-in parking scenarios, and shows that the RL planner achieves state-of-the-art performance over Hybrid A* while offering real-time inference and reduced reliance on localization/tracking. The work includes real-vehicle deployment experiments and discusses limitations and future directions, including automatic curricula.

Abstract

Real-time path planning in constrained environments remains a fundamental challenge for autonomous systems. Traditional classical planners, while effective under perfect perception assumptions, are often sensitive to real-world perception constraints and rely on online search procedures that incur high computational costs. In complex surroundings, this renders real-time deployment prohibitive. To overcome these limitations, we introduce a Deep Reinforcement Learning (DRL) framework for real-time path planning in parking scenarios. In particular, we focus on challenging scenes with tight spaces that require a high number of reversal maneuvers and adjustments. Unlike classical planners, our solution does not require ideal and structured perception, and in principle, could avoid the need for additional modules such as localization and tracking, resulting in a simpler and more practical implementation. Also, at test time, the policy generates actions through a single forward pass at each step, which is lightweight enough for real-time deployment. The task is formulated as a sequential decision-making problem grounded in a bicycle model dynamics, enabling the agent to directly learn navigation policies that respect vehicle kinematics and environmental constraints in the closed-loop setting. A new benchmark is developed to support both training and evaluation, capturing diverse and challenging scenarios. Our approach achieves state-of-the-art success rates and efficiency, surpassing classical planner baselines by +96% in success rate and +52% in efficiency. Furthermore, we release our benchmark as an open-source resource for the community to foster future research in autonomous systems. The benchmark and accompanying tools are available at https://github.com/dqm5rtfg9b-collab/Constrained_Parking_Scenarios.

Figures (10)

• Figure 1: Non-optimal paths generated by the Hybrid A* planner in constrained parking environments. The start pose is denoted by magenta rounded rectangles, and the target pose by cyan rounded rectangles. Planned paths are shown in blue, with shaded gray regions indicating the space occupied by the vehicle along its intermediate poses. Due to partial observability, the current solutions for both cases are likely to result in collisions (obstacles are denoted by solid red lines).
• Figure 2: Challenges in spawning ego initial poses in the sparse obstacle representation environment and our rollout solution. Arrows in (b) represent the heading directions, respectively.
• Figure 3: Our feature extractor architecture for vectorized environment observations.
• Figure 4: Overview of our closed-loop path generation method. The simulator is initialized with a realistic parking scenario, and the environment is iteratively updated based on the RL policy. This framework enables both training the policy and extracting planned paths during inference.
• Figure 5: First seven stages in the curriculum learning. Stage 1 and 2 directly inherit the heading angle from rollout results. For other stages, the ego heading is reset to a collision-free angle sampled from the stage-specific range. All seven stages, the lateral offset is taken from the corresponding rollout result.