Modeling Pedestrian Crossing Behavior: A Reinforcement Learning Approach with Sensory Motor Constraints

TL;DR

This work tackles safe autonomous-vehicle interactions with pedestrians by proposing a constrained reinforcement learning framework that embeds sensory-motor realism, including noisy perception, looming aversion, time pressure, walking effort, and ballistic speed control, plus a biomechanical walking model. It uses a human-in-the-loop dataset from a controlled crossing experiment and optimizes non-policy parameters via Bayesian optimization, while policy learning is conducted with Proximal Policy Optimization in a PBOMDP setting. The results show that the fully sensory-motor (SM) variant reproduces a broad set of human-like behaviors, notably gap acceptance, CIT, day/night differences, and realistic walking speed profiles, underscoring the importance of integrating sensory-motor constraints for realism and generalization. The findings have practical implications for improving AV decision-making, virtual testing, and traffic-safety modeling, by providing a more accurate and interpretable framework for pedestrian-vehicle interactions.

Abstract

Understanding pedestrian behavior is crucial for the safe deployment of Autonomous Vehicles (AVs) in urban environments. Traditional pedestrian behavior models often fall into two categories: mechanistic models, which do not generalize well to complex environments, and machine-learned models, which generally overlook sensory-motor constraints influencing human behavior and thus prone to fail in untrained scenarios. We hypothesize that sensory-motor constraints, fundamental to how humans perceive and interact with their surroundings, are essential for realistic simulations. Thus, we introduce a constrained reinforcement learning (RL) model that simulates the crossing decision and locomotion of pedestrians. It was constrained to emulate human sensory mechanisms with noisy visual perception and looming aversion. Additionally, human motor constraint was incorporated through a bio-mechanical model of walking. We gathered data from a human-in-the-loop experiment to understand pedestrian behavior. The findings reveal several phenomena not addressed by existing pedestrian models, regarding how pedestrians adapt their walking speed to the kinematics and behavior of the approaching vehicle. Our model successfully captures these human-like walking speed patterns, enabling us to understand these patterns as a trade-off between time pressure and walking effort. Importantly, the model retains the ability to reproduce various phenomena previously captured by a simpler version of the model. Additionally, phenomena related to external human-machine interfaces and light conditions were also included. Overall, our results not only demonstrate the potential of constrained RL in modeling pedestrian behaviors but also highlight the importance of sensory-motor mechanisms in modeling pedestrian-vehicle interactions.

Figures (6)

• Figure 1: Panel (a) is the Model framework. Panel (b) is the virtual environment. Panel (c) is a schematic of the deceleration procedure used in this study. Panel (d) is the walking model, adopted from carlisle2023optimization
• Figure 2: Gap acceptance rate in non-yielding scenarios and early crossing rate in yielding scenarios as observed in human behavior and predicted by the models.
• Figure 3: Crossing Initiation Time (CIT) under different conditions. This figure shows the distribution of CIT under different conditions. The first row shows human data, the second row presents the VLW model result (accounting for walking effort), and the third row shows the VL model result (without walking effort). The blue lines within each violin plot represent the mean values. Notably, in the late crossing category for the VL model, no CIT data is available for a 5-second time gap at 30 mph, indicating that no agents chose to cross late under this condition.
• Figure 4: Average walking speed. This figure illustrates the average walking speeds for different crossing decisions—non-yielding, early crossing, and late crossing—under different vehicle speeds (25 mph and 30 mph) and eHMI conditions (on and off). Each pair of violins represents different vehicle speeds (25 mph and 30 mph) within the same time gap and eHMI on/off condition. The black dashed lines indicate the mean walking speed for each pair of violins. This pairing method is used because vehicle speed did not have a significant effect on walking speed in the experiment.
• Figure 5: Average walking speed profile of the experiment, and different model variants. gray lines are the randomly selected speed curves and black lines are the averaged speed curves of all conditions. The blue dashed lines denote the road curbs, and the walking direction is from left to right.