Realistic adversarial scenario generation via human-like pedestrian model for autonomous vehicle control parameter optimisation

TL;DR

The paper addresses the challenge of safely evaluating autonomous vehicles in mixed traffic by introducing a cognitively inspired, human-like pedestrian model (COMMOTIONS) into CARLA, paired with adversarial scenario generation and controller optimisation. By comparing against a rule-based CARLA pedestrian and two AV controllers, the study shows that realistic pedestrian models produce more plausible interactions (e.g., increasing gap acceptance with time-to-arrival and smoother decelerations) and reveal safety-critical events driven by inter- and intra-individual variability. Adversarial scenarios are generated via Bayesian optimisation over time-to-arrival to identify low-time-to-encroachment PET values, forming a basis for three scenario sets used to optimise braking distance. Optimising braking distance on these realistic adversarial scenarios improves safety and efficiency, outperforming baselines that rely on random or overly aggressive adversaries. The results support a scalable, human-centred simulation pipeline for credible AV evaluation and behaviorally informed controller tuning.

Abstract

Autonomous vehicles (AVs) are rapidly advancing and are expected to play a central role in future mobility. Ensuring their safe deployment requires reliable interaction with other road users, not least pedestrians. Direct testing on public roads is costly and unsafe for rare but critical interactions, making simulation a practical alternative. Within simulation-based testing, adversarial scenarios are widely used to probe safety limits, but many prioritise difficulty over realism, producing exaggerated behaviours which may result in AV controllers that are overly conservative. We propose an alternative method, instead using a cognitively inspired pedestrian model featuring both inter-individual and intra-individual variability to generate behaviourally plausible adversarial scenarios. We provide a proof of concept demonstration of this method's potential for AV control optimisation, in closed-loop testing and tuning of an AV controller. Our results show that replacing the rule-based CARLA pedestrian with the human-like model yields more realistic gap acceptance patterns and smoother vehicle decelerations. Unsafe interactions occur only for certain pedestrian individuals and conditions, underscoring the importance of human variability in AV testing. Adversarial scenarios generated by this model can be used to optimise AV control towards safer and more efficient behaviour. Overall, this work illustrates how incorporating human-like road user models into simulation-based adversarial testing can enhance the credibility of AV evaluation and provide a practical basis to behaviourally informed controller optimisation.

Figures (6)

• Figure 1: Illustration of the zebra crossing interaction scenario used to evaluate the impact of different pedestrian models on AV behaviour. The pedestrian begins at a standstill, positioned 4 m from the centreline of the vehicle lane and 2 m from the kerb, while the AV approaches at the constant speed under different Time-to-Arrival (TTA) settings. The pedestrian began at a standstill, positioned 4 m from the centreline of the vehicle lane and 2 m from the kerb, and waited at the kerb before initiating a crossing
• Figure 2: Comparison of AV-pedestrian interaction metrics across different pedestrian and AV model combinations at varying Time-to-Arrival (TTA) values. The three panels from left to right show: collision rate, gap acceptance rate, and sudden speed change rate.
• Figure 3: Distribution of post-encroachment times (PET) across pedestrian-AV combinations and TTA values. The red dashed line marks the 1.5 s safety-critical threshold.
• Figure 4: Comparison of pedestrian-vehicle trajectory patterns across different combinations of AV controllers and pedestrian models. Each line represents one interaction, plotted by the vehicle’s and pedestrian’s distances to the crossing point.
• Figure 5: Post-encroachment time (PET) versus Time-to-Arrival (TTA) for the 100 individual pedestrians from the COMMOTIONS model under their respective low-PET individual-TTA combinations prior to optimisation. Markers show the minimum PETs for each individual, and marker colour indicates the order of crossing. Each coloured line shows the mean PET across TTAs for one of the eight pedestrians with PET below the safety threshold of 1.5 s (dashed line), and the shaded envelopes denote the range across repeated runs for these pedestrians, to illustrate stochastic intra-individual variability.