UniUncer: Unified Dynamic Static Uncertainty for End to End Driving

TL;DR

UniUncer is presented, the first lightweight, unified uncertainty framework that jointly estimates and uses uncertainty for both static and dynamic scene elements inside an E2E planner and design an uncertainty-aware gate that adaptively modulates reliance on historical inputs based on current uncertainty levels.

Abstract

End-to-end (E2E) driving has become a cornerstone of both industry deployment and academic research, offering a single learnable pipeline that maps multi-sensor inputs to actions while avoiding hand-engineered modules. However, the reliability of such pipelines strongly depends on how well they handle uncertainty: sensors are noisy, semantics can be ambiguous, and interaction with other road users is inherently stochastic. Uncertainty also appears in multiple forms: classification vs. localization, and, crucially, in both static map elements and dynamic agents. Existing E2E approaches model only static-map uncertainty, leaving planning vulnerable to overconfident and unreliable inputs. We present UniUncer, the first lightweight, unified uncertainty framework that jointly estimates and uses uncertainty for both static and dynamic scene elements inside an E2E planner. Concretely: (1) we convert deterministic heads to probabilistic Laplace regressors that output per-vertex location and scale for vectorized static and dynamic entities; (2) we introduce an uncertainty-fusion module that encodes these parameters and injects them into object/map queries to form uncertainty-aware queries; and (3) we design an uncertainty-aware gate that adaptively modulates reliance on historical inputs (ego status or temporal perception queries) based on current uncertainty levels. The design adds minimal overhead and drops throughput by only $\sim$0.5 FPS while remaining plug-and-play for common E2E backbones. On nuScenes (open-loop), UniUncer reduces average L2 trajectory error by 7\%. On NavsimV2 (pseudo closed-loop), it improves overall EPDMS by 10.8\% and notable stage two gains in challenging, interaction-heavy scenes. Ablations confirm that dynamic-agent uncertainty and the uncertainty-aware gate are both necessary.

Figures (5)

• Figure 1: Comparison between (a) a standard E2E autonomous driving model and (b) our uncertainty-aware E2E model. Our model introduces uncertainty estimation in both static and dynamic branches, and employs an uncertainty-aware gating mechanism to modulate historical information based on current uncertainty levels.
• Figure 2: Overview of the UniUncer framework. UniUncer extends the static and dynamic branches with a unified uncertainty estimation head to predict Laplace parameters. These parameters are encoded and fused with the original queries to form uncertainty-aware queries, which serve as input to the planner and gate historical information (e.g., ego state and temporal queries) via an uncertainty-aware gate.
• Figure 3: Visualization of the Uncertainty-aware Gate (fine-grained feature- time matrix mode) for Ego Status. The upper row illustrates a complex scenario, while the lower row corresponds to a simple scenario. Driving commands include L (left turn), S (keep straight), R (right turn), and NA (not available). Vx and Vy denote velocities along the $x$- and $y$-axes, respectively, while ax and ay represent accelerations along the $x$- and $y$-axes.
• Figure 4: Qualitative results on the nuScenes dataset. With unified uncertainty estimation, the predicted trajectory from our model is closer to the ground-truth trajectory compared to the baseline model. Ellipses on map vertices and dynamic objects represent the scale parameter $b$ along the $x$- and $y$-axes. For dynamic objects, the uncertainty is lower on the visible side of the vehicle compared to the invisible side. Best viewed in the digital version.
• Figure 5: Qualitative results on the Navhard two-stage test. Uncertainty estimation enables the planner to generate trajectories that are closer to the ground-truth (GT) trajectory in general. It demonstrates that uncertainty estimation for dynamic agents is critical for collision avoidance in challenging pseudo closed-loop scenarios rendered by 3DGS, which can introduce synthetic artifacts (e.g., the bus and vehicle in the second row). By assigning higher uncertainty to such potentially unreliable detections, the planner adopts a more conservative strategy, successfully avoiding over-reaction and improving robustness. Better view in digital version.